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ORGANIZATIONAL ANALYSIS OF THE UNITED STATES ARMY 

EVALUATION CENTER 


ABSTRACT 


This study of the U.S. Army Evaluation Center (AEC) used an organizational systems 
framework to analyze factors related to strategy structure, processes and results 
experienced at AEC during fiscal year 2013. The researcher’s experience, coupled with 
existing survey data collected from established questionnaires, interviews and 
authoritative information sources, was used to analyze AEC as a system. 

The Organizational Systems Eramework model used for this Joint Applied Project 
served as an excellent diagnostic tool to identify improvements to increase efficiency and 
effectiveness. Organization system analysis using the OSE model was successful in 
providing a baseline and key information required to design AEC for the future. It is 
recommended that AEC continue using the OSE to identify future improvements; focus 
on the factors that are within AEC’s control to change (i.e., throughput factors) and focus 
on the factors with the greatest improvement potential. The organizational analysis 
showed that AEC achieves a “fairly strong” level of congruence between the inputs, 
throughputs and results. However, there are two areas where congruency among the 
factors is assessed as “weak,” and 19 areas where congruency among the factors is 
assessed as “average.” Recommendations to improve organizational performance were 
provided as a result of the analysis. 

Although this research was successful in analyzing AEC as a system, many of the 
findings, recommendations, and conclusions drawn in this paper warrant dedicated and 
more in-depth quantitative analysis or consideration from different perspectives. 
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I. 


INTRODUCTION 


A. THESIS OVERVIEW 

This joint applied project conducted an organizational analysis of the U.S. Army 
Evaluation Center (AEC) to describe how external and internal organizational factors 
impact AEC performance. AEC performance is in terms of organizational efficiency 
and effectiveness. Organizational efficiency is defined as the ratio of inputs to outcomes 
in the organization’s transformation process (McShane & Glinow, 2009, p. 329). 
Operational effectiveness is a broad concept which includes the organization’s fit with 
the external environment, internal subsystems configuration for high-performance, 
emphasis on organizational learning and the ability to satisfy the needs of the key stake 
holders (McShane & Glinow, 2009, p. 329). General systems theory and, in particular, an 
organizational systems model, provided the theoretical foundation for drawing 
conclusions and making recommendations concerning complex organizational behaviors. 
It shows the interrelationships between all of the factors that influence an organization. 
The approach assumes that that an organization can only be understood by examining the 
sum of all parts and at the level of congruence between them. Congruence is the degree to 
which the system components interact and create interdependencies between parts 
(Nadler & Tushman, 1980). The overall purpose was to analyze incongruence among key 
organizational variables and determine impact on mission. The intent was to assist 
leaders, managers, and practitioners in ways to improve the fit among relevant variables, 
thereby improving system or organizational performance. 

B. METHODOLOGY 

1. Research Question 

The primary research question of this Joint Applied Project is “How can an 
organizational systems analysis be used to baseline the Army Evaluation Center and 
provide leadership with key information required to better design AEC for the future?” A 
greater understanding of an organization as a system is empowering to leaders. Analyzing 
an organization through a systems approach encourages practitioners to examine 
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interdependencies among the organization and environmental factors in a deliberate 
manner. It is essential to understand these interdependencies among variables, 
congruence or the relative “fit” of variables determines performance. The model is about 
cause-and-effect relationships, which may be far apart in time and/or location. 

Secondary research questions focus on the congruence among key organizational 
variables and determine impact on mission: To what extent are the current organization 
framework factors congruent? The secondary research questions focus on the 
relationships amongst the factors and include: 

To what extent are the inputs factors congruent with the throughput factors? 

To what extent are the throughput factors congruent with each other? 

To what extent are the throughput factors congruent with the outputs? 

2. Literary Review 

Articles and notes from courses attended at the Naval Postgraduate School were 
reviewed throughout the development of this paper, and those utilized are referenced. 
Additionally, summary data from various briefing slides, information papers and policies 
were referred to and referenced. Multiple articles located on the World Wide Web were 
reviewed and referenced. Additional information stemmed from the researcher’s personal 
observations. 


3. Description of the Organizational Systems Framework (OSF) Model 

A description of AEG is provided in Chapter II using the OSF model to describe 
the organization as a system. Robert’s OSF model was derived from the basic system 
model of inputs, processes and outputs (Roberts, 2000). Nadler and Tushman’s 
congruence theory of organizations was applied to Robert’s OSF model factors (Nadler & 
Tushman, 1980). Based on the model, the description is organized into three major 
subjects. 


a. Inputs 

Inputs are external influences or factors fed into the system. They may include 

raw data or pre-existing data provided by the external system to include: 
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• Environmental factors, such as political, economic, social, and 
technological forces or trends; 

• Key factors for the organization to be successful; and 

• System direction, to include its mission, vision, goals, strategic issues, and 
mandates. 

b. Throughputs 

Throughputs are factors involved with the transformation of input into output 
(also referred to as design factors). In this model, they include: 

• Tasks-The basic tasks, jobs or core competencies of the organization; 

• Technology-The condition of the facilities and equipment, work flow, 
activities involved in the work flow, etc.; 

• Structure-The organization chart reflecting groupings of people, how 
tasks and/or roles are combined, etc.; 

• People-Types of people making up the organization, types of experiences, 
skills, knowledge and abilities, motivational factors, etc.; and 

• Processes-Planning, communication, human resource management, 
training plans, acquisition and contracting, etc. 

c. Results 

Results are intentional and unintentional end products of the system. They 
include: 

• Culture-Includes the behavioral norms and values, how conflict is 
managed, impact of culture on the organization, informal patterns of 
interaction, etc.; 

• Outputs-Results of the process on the input. This includes what the system 
has to offer (products or services), how they are measured, and indicators 
of performance; and 

• Outcomes-How the outputs are viewed in terms of the environment and 
the consequences to the stakeholders. 

To better understand the three main components of the OSF model, an illustration 
is provided in Figure 1 (Roberts, 2000). This figure serves as a ready reference for the 
reader throughout the paper. 
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Figure 1. Organizational Systems Framework Model (from Professor Nancy Roberts, Naval Postgraduate 

School, 2000) 
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4. Chapter overview 

Chapter II provides a background of the Defense Acquisition System, DOD test 
and evaluation and the Army test and evaluation organizations and their purposes. 
Chapter III describes AEC in terms of the OSF. Chapter IV presents the analysis of 
implementing the OSF by assessing the congruence between the inputs and throughputs, 
the throughputs and throughputs and the throughputs and results. Chapter V documents 
the findings and recommendations. Conclusions are found in Chapter VI Conclusions. 
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II. BACKGROUND 


A. DEFENSE ACQUISITION SYSTEM 

The Defense Acquisition System is the management process by which DOD 
develops and buys weapons and other systems. It is governed by Directive 5000.01, The 
Defense Acquisition System, and Instruction 5000.02, Operation of the Defense 
Acquisition System, and utilizes the procedures described in the Defense Acquisition 
Guidebook. The primary objective of Defense acquisition is to acquire quality products 
that satisfy user needs with measurable improvements to mission capability and 
operational support, in a timely manner, and at a fair and reasonable price 
(Undersecretary of Defense (AT&L), 2003). 

The generic model for the Defense Acquisition Management System is shown in 
Figure 2. The life cycle process consists of periods of time called phases separated by 
decision points called milestones. Some phases are divided into two efforts separated by 
program reviews. These milestones and other decision points provide both the PM and 
milestone decision authorities (MDAs) the framework with which to review acquisition 
programs, monitor and administer progress, identify problems, and make corrections. The 
MDA will approve entrance into the appropriate phase or effort of the acquisition process 
by signing an acquisition decision memorandum upon completion of a successful 
decision review (Undersecretary of Defense (AT&L), 2008). 
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Figure 2. Defense Acquisition Management System (from Undersecretary 

of Defense (AT&L), 2008) 


B. TEST AND EVALUATION 

“The primary purpose of test and evaluation (T&E) is to support system 

development and acquisition by serving as a feedback mechanism in the iterative systems 
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engineering process” (United States Army, 2006). The standard T&E process currently 
used to support the acquisition of materiel is described in the DOD Defense Acquisition 
Guidebook (Department of Defense, 2013) and Army Regulation 73-1, Test and 
Evaluation Policy. The “product” of the Army T&E process is an understanding of 
system capabilities, which is documented in integrated (developmental and operational) 
evaluations used to inform production and fielding decisions. This process consists of the 
collection of data from tests. Modeling and Simulation (M&S), demonstrations and 
experiments in order to evaluate the technical performance, operational effectiveness, 
suitability, and survivability of the system under development. The purpose of T&E 
during the development and acquisition of a defense system is to identify and understand 
the areas of risk that must be accepted, reduced, or eliminated (Department of Defense, 
2012, p. 23). 

1. “Test” and “Evaluation” 

a. Test 

Test denotes any program or procedure that is designed to obtain, verify, or 
provide data for the evaluation of any of the following: (1) progress in accomplishing 
developmental objectives; (2) the performance, operational capability, and suitability of 
systems, subsystems, components, and equipment items; and (3) the vulnerability and 
lethality of systems, subsystems, components, and equipment items (Department of 
Defense, 2012, p. 77). 

b. Evaluation 

Evaluation denotes the process whereby data are logically assembled, analyzed, 

and compared to expected performance to aid in systematic decision making. It may 
involve review and analysis of qualitative or quantitative data obtained from design 
reviews, hardware inspections, M&S, hardware and software testing, metrics review, and 
operational usage of equipment (Department of Defense, 2012, p. 77). 
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c. Test and Evaluation 

T&E is a process by which a system or components are tested and results 
analyzed to provide performance related information. This information has many uses, 
including risk identification and mitigation as well as providing empirical data to validate 
models and simulations. T&E enables an assessment of the attainment of technical 
performance, specifications, and system maturity to determine whether systems are 
operationally effective, suitable, and survivable for their intended use. There are three 
distinct types of T&E defined in statute or regulation: Developmental Test and evaluation 
(DT&E), Operational Test and Evaluation (OT&E), and Eive Eire Test and Evaluation 
(EET&E) (Department of Defense, 2012, p. 77). 

The types and tasks of T&E as defined by the DAU Program Managers Tool Kit 
are shown in Table 1. 


Developmental T&E (DT&E) 

Operational T&E (OT&E) 

Technical performance measurement 

Operational effective/suitable/survivable 

Developmental agency responsible (PM) 

Operational Test Agency (OTA) responsible 

Technical personnel 

“Typical” user personnel 

Limited test articles/each test 

Many test articles/each test 

Controlled environment 

“Combat” environment/threats 

Components, sub-systems, assemblies, systems 

“Production Rep” test articles 

Contractor involved 

Contractor may not be allowed (lOT&E) 


Table 1. Comparison of T&E Types and Tasks (from Parker, 2011 p. 62) 


EET&E is the term defined as “Major systems and munitions programs: 
survivability testing and lethality testing required before full-scale production.” (10 U.S. 
Code 2355). EET&E addresses two distinct types of testing-survivability and lethality 
(Department of Defense, 2012, p. 33). 

2. Test and Evaluation Oversight. 

The DOD organization for the oversight of T&E is illustrated in Eigure 3 


(Department of Defense, 2012, p. 10). Eor the USD(AT&E), DT&E oversight is 
performed by the DASD(DT&E), within the Office of the Assistant Secretary of Defense 
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for Research and Engineering (ASD(R&E)). The DOT&E provides OT&E oversight for 
the SecDef. 
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Figure 3. DOD T&E Organizations (from Department of Defense, 2012, p. 
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C. ARMY TEST AND EVALUATION 

1. Army Acquisition Executive (AAE) 

The Assistant Secretary of the Army for Acquisition, Logistics, and Technology 

(ASA(ALT)) has the principal responsibility for all Department of the Army matters and 

policy related to acquisition, logistics, technology, procurement, the industrial base, and 

security cooperation. Additionally, the ASA(ALT) serves as the AAE. The AAE 

administers acquisition programs by developing/promulgating acquisition policies and 

procedures as well as appointing, supervising, and evaluating assigned program executive 

officers (PEOs) and direct-reporting PMs. The AAE serves as the Milestone Decision 

Authority on ACAT IC and ACAT lAC programs (Department of Defense, 2012, p. 15). 

10 









2. Army T&E Executive 

The Army T&E Executive establishes; reviews, enforces, and supervises Army 
T&E policy and procedures including overseeing all Army T&E associated with the 
system research, development, and acquisition of all materiel systems and C4/IT systems. 
As delegated by the AAE, the Army T&E Executive is the sole Headquarters, 
Department of the Army (HQDA) approval authority for TEMPs. 

The Test and Evaluation Office within the Office of the Deputy Under Secretary 
of the Army, known as the Deputy Under Secretary of the Army for Test and Evaluation 
(DUSA-TE), provides support for the Army T&E Executive. In this capacity, it has the 
mission to establish policy and resources that are disciplined and flexible enough to 
support safe and reliable equipment for the current and future Army and DOD chemical 
and biological defense. DUSA-TE also provides T&E subject matter expertise and 
oversight of Army and DOD chemical and biological acquisition programs and represents 
Army T&E interests at OSD and tri-Service committees and forums (Department of 
Defense, 2012, p. 15-16). 

3. Army Test and Evaluation Command (ATEC) 

The Army is unique among the services in having a single organization. Army 
Test and Evaluation Command (ATEC), which is responsible for developmental testing, 
operational testing, and the continuous (through all phases of a program’s life cycle) 
integrated (developmental and operational) evaluation of materiel. 

The ATEC commander is a major general who reports directly to the Vice Chief 
of Staff of the Army through the Director of the Army Staff (Department of the Army, 
2006, p. 9). ATEC is comprised of subordinate commands. The Army Evaluation Center 
(AEC), headquartered at Aberdeen Proving Ground, Maryland, develops evaluation 
plans, determines data requirements and sources (analysis, developmental testing, 
operational testing, M&S, exercises), observes testing, and evaluates system 
effectiveness, suitability, and survivability (Department of the Army, 2006, p. 10) . AEC 
also provides testers with a safety release for systems before the start of pretest training 
for tests that use soldiers as test participants per AR 385-1 The Army Safety Program 
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and provides safety confirmations for milestone decision review and the materiel release 
decision (Department of the Army, 2013, pp. 8-9). The Operational Test Command, 
headquartered at Fort Hood, Texas, manages operational test centers throughout the U.S. 
and plans, conducts, and reports on operational tests (Department of the Army, 2006, 
p. 10). Six developmental test centers located throughout the U .S. plans, conducts, and 
reports on developmental tests: White Sands Missile Range, New Mexico; Aberdeen Test 
Center at Aberdeen Proving Ground, Maryland, Dugway Proving Ground (West Desert 
Test Center) at Dugway Proving Ground, Utah; Electronics Proving Ground at Fort 
Huachuca, Arizona; Redstone Test Center at Redstone Arsenal, Alabama; and Yuma 
Proving Ground (Cold Regions Test Center, Tropic Regions Test Center, Yuma Test 
Center) at Yuma Proving Ground, Arizona (Department of the Army, 2006, p. 10). ATEC 
organizational structure is shown in Figure 4. 



WDTC CRTC 


WDTC - West Desert Test Center 
CRTC - Cold Regions Test Center 
TRTC-Tropic Regions Test Center 
YTC - Yuma Test Center 

Figure 4. ATEC Organization (from Army Evaluation Center, 2011) 
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Also unique among the services is the fact that ATEC, as the Army’s Operational 
Test Agency, is responsible for defining LFT&E requirements and reporting on EET&E 
results (program managers assume this responsibility in other services). The unique 
characteristics of ATEC activities were endorsed by a 1999 Defense Science Board 
recommendation, which implicitly urged the other services to adopt the Army/ATEC 
model: 


Each of the Service DT&OT organizations should be consolidated, to 
include integrated planning, use of models, simulation and data reduction. 
Planning should be totally integrated, and the OSD T&E organizations 
consolidated. There should be integrated use of models, simulation and 
data reduction. Except for limited dedicated Operational Test and 
Evaluation (OT&E), contractor and government testing should also be 
integrated. (OSD AT&E, 1999, p. 3) 
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III. DESCRIPTION OF AFC AS A SYSTEM 
USING THE OSF MODEL 


The purpose of this section is to describe the AEC as a system using the 
Organizational Systems Framework (OSF) model. A system is defined as a set of 
interrelated components working towards a common purpose. The model is based on the 
concept of inputs, throughputs, and results. The input is what is received from the 
external environment and the output is what leaves the system, returning back into the 
environment. The transformation of the input by the system to an output is called the 
throughput (McShane & Glinow, 2009, pp. 6-7). 

The basic systems approach to organizations acknowledges the existence of open 
systems; meaning they interact with other systems outside of themselves and this 
interaction includes inputs (what enters the system from outside) and outputs (what 
leaves the system for the environment). The OSF model breaks down these two 
components into subcategories and includes the throughput, which occurs between the 
inputs and outputs. As mentioned previously, inputs include the external environment, 
system direction, and key success factors. Throughput, referred to as Design Factors in 
the OSF, consists of tasks/jobs, technology, structure, people, and processes or 
subsystems. Culture, outputs, and outcomes make up the ‘results’ portion of the model 
(Roberts, 2000). To gain insight into AEC, the OSF model is applied based on the 
researcher’s close experience working in the organization, briefing slides and other 
applicable documents from the center. 

A. INPUTS 

1. External Environment 

The United States Army has endured a perpetual cycle of sustained combat 
operations for over 10 years. This, coupled with the austere fiscal climate, presents a 
challenging stage. In the OSF model, the external influences are categorized as political, 
technological, social, and economic factors which make up the environment where the 
system or organization exists (Roberts, 2000). 
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a. Political Forces 

Army Secretary John McHugh commissioned a review and analysis of Army 
acquisition to determine lessons learned from past acquisition failures. Published in 
January 2011, the DeckerAVagner Task Force report: “Army Strong: Equipped, Trained 
and Ready: The Final Report of the Army Acquisition Review” reported the Army 
terminated 22 Major Defense Acquisition Programs (MDAP) of record before completion 
from 1990 to 2010 (Department of the Army, 2011, p. viii). Cancellations, schedule 
slippages, cost over-runs and failure to deliver timely solutions to the operators’ 
requirements have caused Army leadership. Army leadership, OSD, Capitol Hill and 
industry to lose trust in the Army’s acquisition processes and capability (Department of 
the Army, 2011, p. ix). The erosion in the core competencies of the personnel responsible 
for the development of requirements and the acquisition of systems and services, have 
exasperated the issue (Department of the Army, 2011, p. ix). 

All acquisition activities can be impacted when changes in control of the 
Congress as a result of elections. The effects of elections are difficult to predict; but at a 
minimum, funding priorities will be reviewed and past decisions, positively or negatively 
affecting acquisition programs, could be revisited. 

Congress requires the DOD to provide the following reports that include 
information on T&E: 


• Selected Acquisition Report (SAR). This report consists of cost, schedule, 
and performance data. The SAR describes Acquisition Category (ACAT) I 
system characteristics required and outlines significant progress and 
problems encountered. It lists tests completed and issues identified during 
testing (10 U.S. Code § 2432). 

• Director, Operational Test & Evaluation (DOT&E) Annual Report. This 
report is provided by the DOT&E to the Secretary of Defense and the 
committees on Armed Services, National Security, and Appropriations. 
The report provides a narrative and resource summary of all operational 
test and evaluation (OT&E) to include live-fire testing (EFT) and related 
issues, initiatives, other interest areas, activities, and assessments in the 
previous fiscal year (10 U.S. Code § 139). 
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• Beyond Low-Rate Initial Production (BLRIP) Report. Before proceeding 
to BLRIP for each major defense acquisition program (MDAP), the 
DOT&E must report to the SecDef, Deputy Secretary of Defense, Under 
Secretary of Defense for Acquisition, Technology and Logistics 
(USD(AT&L)), Secretaries of the Military Departments, and Congress. 
This report addresses the adequacy of the Service initial operational test & 
evaluation (lOT&E) and whether the T&E results confirm that the tested 
item or component is effective, suitable, and survivable for combat. When 
oversight of live-fire test & evaluation (EET&E) was moved to the 
DOT&E, the EET Report was added to the BERIP report content (10 U.S. 
Code § 2399). 

• Eoreign Comparative Testing (ECT) Report. The USD(AT&E) should 
notify Congress a minimum of 30 days prior to the commitment of funds 
for initiation of new ECT evaluations of equipment produced by select 
allied and friendly foreign countries (10 U.S. Code § 2350a.(g)). 

• Joint Deputy Assistant Secretary of Defense for Developmental Test & 
Evaluation (DASD(DT&E)) and Deputy Assistant Secretary of Defense 
for Systems Engineering (DASD(SE)) Annual Report. This report is 
required by statute to be provided to the committees on Armed Services 
and Appropriations. The joint report includes the significant 
Developmental Test and Evaluation (DT&E) and systems engineering 
(SE) activities for the Department’s MDAPs, major automated 
information systems (MAIS), and special interest programs. The report 
evaluates the progress of weapon systems’ performance for programs 
designated for OSD T&E oversight (10 U.S. Code § 139). 


b. Economic 

(1) Department of Defense Budget 

The economic component of the external environment is defined by the researcher 
as the national economy and the fiscal health of our nation. Although the political 
priorities identify where the spending goes, the economic component plays a significant 
role in determining how much there is to spend. The base budget of the U.S. Department 
of Defense (DOD) increased about 40 percent in real terms from 2001 to 2012. DOD 
must reduce spending by $487 billion from EY2012-EY2021 in order to comply with the 
Budget Control Act of 2011. (Department of Defense, 2012, p. 1). 
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Figure 5 highlights the difference between the current and previous drawdowns 
(Murdock, 2012). 


Post-Korea 


Post-Vietnam 


Post-Cold War 


Current 

-43% 


-33% 


-36% 


-31% 



Figure 5. Budget Authority Draw-downs (from Murdock, 2012) 


The aggregate impact of inflation in the cost of personnel, health care, operations 
and maintenance (O&M), and acquisitions results in a defense dollar that “buys” less and 
less capability. This internal cost inflation is driving DOD toward a zero-sum trade-off 
tween personnel end-strength and modernization. Operations and maintenance (O&M) 
costs have ballooned over the past few decades. In combination, inflation in the 
personnel, health care, operations and maintenance (O&M) accounts will squeeze 
funding for modernization (procurement and research, development, test, and evaluation 
[RDT&E]) in 2020, as depicted in Figure 6, if current trends are allowed to continue 
(Murdock, 2012). 
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Figure 6. Projected Defense Top Line (from Murdock, 2012) 


(2) AEC Budget 

AEC is funded through the Research Development Test & Evaluation (RDT&E) 
appropriation account. The Defense Acquisition Portal ACQuipedia website describes 
RDT&E as “one of the five major appropriations used by the Department of Defense. 
RDT&E finances research, development, test and evaluation efforts performed by both 
contractors and government installations in the developing equipment, material, or 
computer application software. This includes services (including government civilian 
salaries), equipment, components, materials, end items and weapons used in such 
efforts.” 

Eigure 7 presents AEC’s EY99-EY19 budget profile, as derived from data from 
http://asafm.army.mil/offices/BU/BudgetMat.aspx?OfficeCode=1200, the Army Einancial 
Management Budget Materials website. 
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Budget-Actual & Projected 



Figure 7. AEC Budget - Actual and Projected 


(3) Civilian personnel fringe benefit rates 

Office of Management and Budget (0MB) Circular A-21 requires agencies to use 
standard cost factors to estimate certain costs of government performance. These cost 
factors ensure that specific government costs are calculated in a standard and consistent 
manner to reasonably reflect the cost of performing commercial activities with 
government personnel. Civilian position, full-fringe benefits include four separate 
elements: (1) insurance and health benefits, (2) standard civilian retirement benefits 
(Social Security, Thrift Savings Plan, Federal Employees or Civil Service Retirement 
Systems), (3) Medicare benefits, and (4) miscellaneous fringe benefits. The agency pays 
for salaries and fringe benefits out of their local budget. 0MB determined, based on 
information provided by 0PM, that the civilian position, full-fringe benefit cost factor 
needs to be adjusted upward, from 30.3 percent in PY12 to 30.6 percent in PY13. This 
adjustment is necessary to account for increases in insurance and health benefits and 
civilian retirement benefits. This factor is based only on costs borne by the government 
(not enrollee premiums) and only on behalf of active federal employees (not retirees). 

The DOD Civilian Personnel Eringe Benefits rates are published annually. These 
rates are used when obtaining reimbursement for services provided to agencies outside 
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the Federal Government. The average rate is 27.17 for FY99-FY14. The percentage 
increase from FY99 to FY13 is 21%. (Office of the Under Secretary of Defense 
(Comptroller), 2013). 


c. Technological Advances 

(1) Army Science & Technology Strategy 

The Army Science & Technology (S&T) strategy seeks to develop and mature 
technology that will enable transformational capabilities in the future force while 
pursuing opportunities to accelerate technology maturity for transition into current force 
systems. 

• Force Protection: technologies enable Soldiers and platforms to avoid 
detection, acquisition, and hit, penetration and kill. 

• Intelligence, Surveillance and Reconnaissance (ISR) technologies enable 
persistent and integrated situational awareness and understanding to 
provide actionable intelligence that is specific to Soldier needs across the 
full range of military operations. 

• Command, Control, Communications and Computers (C4) technologies 
provide capabilities for superior decision making, including intelligent 
network decision agents and antennas to link Soldiers and leaders into a 
seamless battlefield network. 

• Lethality technologies enhance Soldiers’ ability and platforms to provide 
overmatch against threat capabilities and include nonlethal technologies 
enabling tailorable lethality options. 

• Medical technologies protect and treat Soldiers to sustain combat strength, 
reduce casualties and save lives. 

• Unmanned Systems technologies enhance the effectiveness of unmanned 
air and ground systems through improved perception, cooperative 
behaviors and increased autonomy. 

• Soldier Systems technologies provide materiel solutions that protect, 
network, sustain and equip Soldiers, and non-materiel solutions that 
enhance human performance. 

• Logistics technologies enhance strategic response and reduce logistics 
demand. 

• Military Engineering and Environment technologies enhance deployability 
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The impact of budget reductions within the Army’s S&T portfolios is unknown. 
The National Military Strategy focuses on operations in the Pacific Rim, introducing 
increased complexity regarding operational environment. Technologies required to 
enhance Programs of Record (PORs) as well as replacing platforms, drives AEC to 
understand the technologies as well as how they will be employed in the operational 
context in order to provide future evaluations. 

(2) Cyber Security 

DOD weapons systems and information technologies operate in an increasingly 
complex, networked, joint information environment. Cyber security considerations 
generally apply to all acquisition systems because they interface to combinations of 
networks, platforms, support systems, or other elements of the operating environment that 
are potentially exploitable by cyber threats that are constantly evolving. 

On January 2, 2013, President Obama signed the 2013 National Defense 
Authorization Act for 2013 (NDAA) into law. Congress included several targeted 
statutory provisions setting federal defense policy on a range of cyber security issues. 
“The Secretary of Defense shall provide to the Committees on Armed Services of the 
House of Representatives and the Senate quarterly briefings on all offensive and 
significant defensive military operations in cyberspace carried out by the [DOD] during 
the immediately preceding quarter,” the NDAA text reads. In a 1 Feb 13 memorandum, 
DOT&E directed the service OTAs to improve information assurance tests (cyber 
security) to be as rigorous and challenging as the cyber threats systems will face 
(Gilmore, 2013). The Army is collectively developing an overarching generic evaluation 
approach and identifying what testing is required from which organization. AEC will 
ultimately evaluate cyber for all covered systems and therefore, is the lead agency for 
developing the cyber evaluation capability. 

(3) Systems of Systems 

The Defense Acquisition Guidebook defines Systems of Systems (SoS) as “a set 
or arrangement of systems that results when independent and useful systems are 
integrated into a larger system that delivers unique capabilities.” Mission-Based T&E 
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methodology was developed to enable robust and systematic SoS T&E. MBT&E focuses 
on the identification and alignment of system components and functions with the tactical 
missions and warfighting functions/tasks that the system supports. The approach 
facilitates testing in an “operationally realistic” environment and evaluating “in the 
mission context at the time of fielding.” Eurther, it facilitates the assessment of system 
functionality, the assessment of the effect of system functionality on operational 
capability, and the assessment of the capability of the warfighter to accomplish mission 
tasks (Wilcox, 2008). 

(4) Network Integration Evaluation 

To mitigate the budget constraints, the Army has constructed a series of Network 
Integration Evaluations (NIEs). The purpose of the NIEs is to work closer with industry 
to expedite the development of new capabilities. The NIEs are semiannual exercises 
conducted at White Sands Missile Range (WSMR), New Mexico, with the purpose of 
placing emerging technologies in the hands of Soldiers to evaluate them in realistic, 
combat-like scenarios. NIEs are used as the operational test venues for some programs of 
record. 

The Agile Process was created to procure and align systems that meet a predefined 
operational need or gap identified for the force. These needs are identified within the Training 
and Doctrine Command (TRADOC) community and fed to the acquisition community, which 
then solicits potential solutions. TRADOC and the Army acquisition community must ensure 
those solutions are aligned to a newly developed or preexisting requirement in order to conduct 
procurement activities within the rules of the Defense Acquisition System (Department of 
Defense Instruction 5000.01/.02). AEC does not have a lead role in the Agile Process however, 
AEC provides safety releases for all systems participating in NIE. 

d. Social Pressures 

The Psychology Dictionary defines social pressures as “the influence that is 
exerted on a person or group by another person or group. It includes rational argument, 
persuasion, conformity and demands” (What is Social Pressure?, n.d.). The Army test and 
evaluation community receives significant and critical attention from the media, which is 
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a form of social pressure on the system. During FY13, ATEC was in the press for 
providing information on Distributed Common Grounds System-Army (DCGS-A) 
(program of record) and Palantir, a rapid initiative funded by the Army’s Rapid Equip 
Eorce (REE). Palantir and DCGS are intelligence-gathering software programs designed 
to store and compute data for many purposes, including predicting lED locations. 
Representative Duncan Hunter was contacted by soldiers in Afghanistan, who relayed 
that the Army denied fielding Palantir as DCGS-A is the Army’s program of record. 
ATEC tested and evaluated DCGS-A and concluded in April 2012 that DCGS-A was 
“overcomplicated, requires lengthy classroom instruction,” and uses an “easily perishable 
skill set if not used constantly.” Palantir was assessed in-theater by the ATEC Eorward 
Operating Assessment team. The assessment report stated, “ninety-six of the 100 
personnel surveyed agreed that Palantir was effective in supporting their mission. The 
overall feedback from the operators and immediate supervisors was that Palantir is a 
user-friendly and reliable program.” The report also recommended purchasing additional 
Palantir systems (Carter, 2014). 

An Army email requested that the original report be destroyed about one month 
after it was released. The report was replaced with a very similar report, minus the 
section recommending the increased purchase of Palantir. The report was corrected as 
ATEC does not recommend quantities of systems. The press perceived that the ATEC 
report was manipulated to prevent units from receiving the Palantir software in favor of 
DCGS-A (Carter, 2014). 

2. Key Success Factors 

According to the OSE model, an element that affects an organization’s future 
success is contained in the question, “What does it take for the organization to be 
successful?” (e.g., what factors are crucial for success). These key factors may change 
from year to year based on the priorities and other external environmental issues. Success 
for the commercial world is straightforward and simple: maximize profit. In turn, this 
means selling products to customers at the right price, right time, and right cost. 
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In AEC system’s case, there is no specific manual or document that explicitly 
calls out key success factors. Success factors support the AEC mission: provide 
information to decision makers. 

Success factors include: 

• Goal attainment-how successful AEC meets its strategic goals and 
objectives 

• Resource utilization-how well AEC uses its available resources to meet 
the mission 

• Adaptability-ability to change to fit with the constantly changing external 
environment. 

3. System Direction 

The system direction acts as the internal compass for the overall systems and 
includes its mission, vision and values. It also emphasizes strategies, goals and any 
mandates levied on the organization. 


a. Mission 

AEC’s mission statement as documented on the AEC web-site is: “To plan, 
support, conduct and provide independent evaluations, assessments, and experiments in 
order to provide essential information to decision-makers” (AEC, 2013). 

b. Vision and Values 

AEC’s vision statement is documented on the web-site: 

AEC exists to support the Army Test and Evaluation Command (ATEC) 
in meeting its responsibilities in defending our country and to help 
improve DOD’s performance and accountability for the benefit of the 
American people. As a sub-command of ATEC, it is our responsibility to 
provide leadership and the customer the most effective, efficient, 
creditable, and reliable information. It is of the utmost importance for our 
organization to reflect excellence in all of our business operations, 
practices and professional endeavors. In our mission to support the 
Department of Defense (DOD), we seek to identify areas for 
improvement, and by doing so we promote the best business practices 
throughout DOD and the Department of the Army (DA). (AEC, 2013) 
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There are no specific AEC values as it draws from the values of the Army: 
loyalty, duty, respect, selfless service, honor, integrity and personal courage. 

c. Strategic Plan-Goals and Objectives 

AEC EY13 Strategic Plan focused on three strategic goals/objectives. 

Goal 1. Organization that is a Great Place to Work 

• Maintain an Organization of Talented Professionals 

• Raise Workforce Credentials and Certifications 

• Ensure good communications and transparency 

• Increase team work 

• Improve workforce quality of life 

Goal 2. Continue to Improve Product Value to our Customers 

• Optimize Tools and Processes to Enhance the Quality of our Products 

• Enhance Relationships with our T&E Partners 

• Eisten to our Customers and Address their Eeedback 

• Work with stakeholder to ensure evaluation plans are adequate and 
efficient 

Goal 3. Ensure the organization is structured for efficient operations 

• Optimize the cost and means of Doing Business 

• Continue to support the ATEC reorganization 

• Ensure organizational roles are refined, while building cohesive, 
integrated teams 

• Creating a flexible organization to respond to changing workload 
environment 

d. Mandates 
Statutes 

Congress was concerned about past abuses where the DOD inappropriately rushed 
systems into production without adequate testing. Table 2 addresses the key statutes in 
Title 10 that specifically address T&E. 
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Topic 

Title 10 Section 

DOT&E access to all OT&E data & records 

§ 139(e)(3) 

DASD(T&E) access to all records and data 

§139b(a)(6) 

Initial OT&E required for combat systems 

§2399(a) 

DASD(T&E) Responsibilities of lead DT&E organization 

§139b(c)(3) 

DOT&E approval of OT&E plan adequacy 

§23990) 

DASD(T&E) approval of DT&E in TEMP 

§139b(a)(5)(B) 

DOT&E Report to Congress before going B-LRIP 

§23990)(3), (4) 

DASD(T&E) DT&E acquisition personnel 

§139b(a)(5)(D) 

DASD(T&E) Assess technological maturity/risk 

§139b(a)(5)(P) 

DOT&E or OTA approval of LRIP quantities 

§2399(c) 

Limits on system contractor involvement 

§2399(d) 

Limit on use of M&S and analysis alone 

§2399(h)(l) 

LET&E of new “covered systems” 

§2366(a) 

LET&E carried out sufficiently early 

§23660) 

LET&E waivers & alternate plans (e.g., for aircraft) 

§2366(c) 

LET&E B-LRIP Report to Congress before going B-LRIP 

§2366(d) 

LET&E of “significant” mods to “covered systems” 

§2366(e) 

Production representative articles for OT&E 

§24000) 

Synopsis Report of EW Programs P.L. 103-160 §220 



Table 2. List of Statutes Governing T&E 


(1) DOD Policy 

OSD expands on Title 10 by imposing additional T&E requirements via CJCS 
3170.01, DODD 5000.1, DODI 5000.2, other DODDs and DODIs, DOT&E and 
DASD(T&E) policy memos. Table 3 shows the list of directives and instructions 
governing T&E. 
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Number 

Title 

CJCS 

3170.01 

Joint Capability Integration and Development System Manual 

DOD 

3235.1-H 

DOD Test and Evaluation of System Reliability, Availability and Maintainability 

A Primer 

DODD 

5000.1 

The Defense Acquisition System 

DODD 

5010.41 

DOD Information Security Program 

DODD 

5I4I.2 

Director of Operational Test and Evaluation (DOT&E) 

DODD 

5200.1 

DOD Information Security Program 

DODI 

5000.2 

Operation of the Defense Acquisition System 

DODI 

5134.17 

Deputy Assistant Secretary of Defense for Developmental Test and Evaluation 
(DASD(T&E)) 

DODI 

5200.40 

DOD Information Technology Security Certification and Accreditation Process 
(DITSCAP) 

DODI 

5230.24 

Distribution Statements on Technical Documents 

DODI 

8500.01 

Cybersecurity 


Table 3. List of DOD, CJCS Directives and Instructions Governing T&E 


DOT&E prescribes DOD OT&E and Eive Eire T&E policy. The Director’s 
reports, by statute, go directly to the Secretary of Defense and Congress. DOT&E current 
initiatives include field new capability rapidly; engage early to improve requirements; 
integrate developmental, live fire, and operational testing; and substantially improve 
suitability before Initial Operational Test & Evaluation (lOT&E). DOT&E is also 
emphasizing T&E of Information Assurance (cyber-security) in Acquisition Programs. 
All DOT&E policies can be retrieved from the Director of Operational Test & Evaluation 
website: http://www.dote.osd.mil/Policy.html and are shown in Table 4. 
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Date 

Subject 

2013 

T&E of Information Assurance in Acquisition Programs 

2012 

Changes to Approved Operational Test Plans 

Independent OT&E Suitability Assessments 

2011 

Timeliness of OT&E Plans 


Clarification of Procedures for OT&E of Information Assurance in Acquisition Programs 


Guidance on the use of DOE in OT&E 


Use of Production-Representative Test Articles for lOT&E 


Designation of Programs for OSD Operational & LET&E Oversight 

2010 

Guidelines for OT&E of Information & Business Systems 


Timely Provision of Test Data 


Standardization of Hard Body Armor Testing 


Timely Analysis & Repotting of T&E Results 


Reporting of OT&E Results 


T&E Initiatives 


Modification of Systems Subject to Survivability Testing Oversight 


Using DOE for OT&E 

2009 

Procedures for OT&E of Information Assurance in Acquisition Programs 

2008 

Definition of Integrated Testing 

2007 

T&E Policy Revisions 

2005 

MOA on Operational Suitability Terminology & Definitions to be used in OT&E 

2002 

Modeling and Simulation 

2000 

Policy on the Use of Test Data in Operational Evaluations 

MOA on MOT&E & JT&E 

1999 

Policy on OT&E of Electromagnetic Environmental Effects & Spectrum Management 

1994 

Software Maturity Criteria for Dedicated OT&E of Software-Intensive Systems 


Table 4. DOT&E Policy (from the Director of Operational Test & 

Evaluation website) 


(2) DASD(DT&E) Policy 

DASD(DT&E) prescribes policy and guidance in support of the acquisition of 
major DOD weapon systems and to provide advocacy, oversight, and guidance to 
elements of the acquisition workforce responsible for DT&E per DODI 5134.17 dated 25 
October 2011. The policy focused to improve the measurable performance criteria and 
associated metrics used to gain insight into DT&E performance; improve the training, 
education, and prestige of the DAWIA T&E acquisition workforce; develop policy and 
guidance to implement recent statutory language; promote the increased use of scientific 
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and statistical T&E methodologies and tools within the acquisition programs; and 
develop DT&E methods for capabilities operating in the cyber domain. 

In EY13, “Shift Eeft” was a major DASD(DT&E) initiative. “Shift Eeft” focuses 
on earlier DT&E activities to identify and fix problems during development when fixes 
are more effective, more efficient and less costly. The initiative has three key focus areas: 
earlier mission context, earlier operability testing and earlier cyber security testing 
(Hutchinson, 2013). 

DASD(T&E) is the T&E functional leader and is designated by the Under 
Secretary of Defense for Acquisition, Technology and Eogistics to improve the 
professional qualification standards for T&E workforce. In coordination with Defense 
Acquisition University (DAU), DASD(T&E) is developing more rigorous qualification 
standards and documentation procedures to track an individual’s demonstrated T&E 
knowledge, skills and experience (Office of the Deputy Assistant Secretary of Defense 
for Developmental Test & Evaluation, n.d.). 

In order to improve test effectiveness and ensure efficient use of scarce resources, 
DASD(T&E) in collaboration with the Commander Air Education and Training 
Command, established the Scientific Test and Analysis Techniques (STAT) Center of 
Excellence (COE). The COE directs the use of scientific and statistical methods in 
developing rigorous test plans and the evaluation of results. Design of Experiments 
(DoE) is one of the tools and techniques utilized for STAT. DoE is a structured process to 
identify metrics, factors and levels that mostly affect effectiveness and suitability (Air 
Eorce Institute of Technology, n.d.). 

(3) HQDA Policy 

As previously discussed, DUSA-TE has the mission to establish policy and is the 
proponent for Army T&E regulations. HQDA T&E Policy is shown in Table 5; Army 
T&E Regulations are shown in Table 6. 
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Proponent 

Policy 

Year 

ASA(ALT) 

Use of Contractor Test Data as an Element of Integrated Test and 

Evaluation 

2012 

ASA(ALT) 

Improving the Reliability of U.S. Army Materiel Systems 

2011 

DUSA(OR) 

TEMP Approval Process Improvements 

2004 

DUSA-TE 

TEMP Policy on Independent Operational Test & Evaluation Suitability 
Assessments and Evaluations 

2012 

DUSA-TE 

Eunding to Assess the Adequacy of Technical Data for Use in Evaluation 

2012 

DUSA-TE 

Documenting Revised T&E Strategies in TEMPs 

2011 

DUSA-TE 

Efficient Use of DOD Test Infrastructure 

2010 

DUSA-TE 

Army Test Synchronization 

2010 

DUSA-TE 

T&E Policy for CBDP Systems 

2007 

TEMA 

Improving HQDA TEMP Approval Process 

2008 

HQDA 

Army Guidelines - Modeling & Simulation in Support of T&E 

2000 


Table 5. HQDA Policy 


(4) Army Policy 


Regulation 


Title 

10-87 


Commands, Army Service Component Commands, and Direct Reporting Units 

25-1 


Army Information Technology 

350-50 


Combat Training Center Program 

385-10 


The Army Safety Program 

40-10 


Health Hazard Assessment Program in Support of the Army Acquisition Process 

525-22 


U.S. Army Electronic Warfare 

700-127 


Integrated Logistics Support 

700-142 


Type Classification, Materiel Release, Eielding and Transfer 

70-1 


Army Acquisition Policy 

70-75 


Survivability of Army Personnel and Materiel 

71-9 


Warfighting Capabilities Determination 

73-1 


Test and Evaluation Policy 

750-10 


Army Modification Program 

750-43 


Army Test, Measurement, and Diagnostic Equipment 


Table 6. Army Regulation Directing T&E 


The Army Test and Evaluation Command (ATEC) is designated as the Army’s 
independent operational test activity by regulation, not statute. Army Regulation 73-1 
Army Test and Evaluation Policy states “USATEC is the Army’s independent 
operational test activity and reports directly to the Vice Chief of Staff, U.S. Army 
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through the Director of the Army Staff.” By means of the U.S. Army Evaluation 
Center— 

1. Perform the duties of a system evaluator for all Army systems except for 
the systems assigned for evaluation to USAMEDCOM, USAINSCOM, and the 
commercial items assigned to USAGE. 

2. Conduct continuous evaluation (CE) on all assigned systems. 

3. Develop and promulgate evaluation capabilities and methodologies. 

4. Coordinate system evaluation resources through the TSARC. (See chap 9.) 

5. Preview programmed system evaluation requirements for possible use of 
M&S to enhance evaluation and reduce costs. 

6. Perform MANPRINT assessments in coordination with Deputy Chief of 
Staff, G-1 (ARE-HRED). 

7. Perform the lES program surveillance for Army systems. Perform 
independent logistics supportability assessments and report them to the Army Eogistician 
and other interested members of the acquisition community. Oversee and evaluate the 
logistics aspects of system acquisition and modification programs and deployed systems 
to ensure supportability. 

8. Participate in program reviews, supportability WIPTs, T&E WIPTs, and 
other working and review groups and in the development of requests for proposal, 
statements of work, and contract data requirements lists. 

With the merger of the U.S. Army Developmental Test Command, AEC is also 
directed to provide testers with a safety release for systems before the start of pretest 
training for tests that use soldiers as test participants; and provide safety confirmations 
for MS decision review and the materiel release decision. 
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(5) ATEC T&E Policy 
ATEC T&E Policy is shown in Table 7. 


Number 

Title 

REG 

73-1 

System Test and Evaluation Policy 

PAM 

73-1 

Volume 1, System Test & Evaluation Procedures 

PAM 

73-1 

Volume 11, Developing, Classifying, and Reporting Test and Evaluation 

Documents 

PB 

2-11 

Organizational Conflicts of Interest (OCI) Involving Contractors in 
Support of ATEC Test and Evaluation (T&E) 


Table 7. ATEC T&E Policy 


ATEC Regulation 73-1 is the primary policy for test and evaluation (T&E) of 
Army materiel and information technology systems. ATEC exercises overall 
management of assigned T&E programs. This regulation addresses guidance for 
developmental testing (DT), operational testing (OT), integrated testing, and system 
evaluation. Department of the Army (DA) officials use ATEC products (plans and 
reports) described in this regulation as input to acquisition decisions. 

ATEC Pamphlet 73-1 Volume I implements ATEC methodology for testing and 
system evaluation in accordance with ATEC Regulation 73-1; provides background 
information on integrated T&E strategies; and provides guidance and suggestions for 
preparing and formatting documentation for tests, evaluations, and assessments 

ATEC Pamphlet 73-1 Volume II is a guide to be used in conjunction with 
applicable regulatory guidance and Volume I of the ATEC Pamphlet 73-1, System Test 
and Evaluation Procedures to ensure ATEC documents and the handling of those 
documents reflect the excellent reputation and credibility of ATEC’s expertise and is 
formal, logically organized, based on independent analysis, relevant in their findings, 
results, recommendations and conclusions, properly marked, safeguarded from 
unauthorized persons, and released by the appropriate approval authority to authorized 
persons only. 

ATEC Policy Bulletin 2-11, Organizational Conflicts of Interest (OCI) Involving 
Contractors in Support of ATEC Test and Evaluation (T&E) dated 19 April 2011 
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establishes policy to ensure that contracts awarded in support of ATEC are free of actual 
or potential OCI and states that contractors may work on Army system development 
programs to the exclusion of participating in ATEC test and evaluation support contracts. 
Alternatively, they may participate in ATEC test and evaluation support contracts to the 
exclusion of working on Army system development programs. This is the only reliable 
and effective means of avoiding violations of Title 10, United States Code, Section 
2399(d). With respect to ATEC test and evaluation service support contracts, contracts 
will not be awarded for operational or developmental test and evaluation support to prime 
contractors or affiliates that are executing Army developmental programs. This is because 
test and evaluation support contractors may be required to evaluate the products and 
services of developers, their subcontractors and suppliers. 

(6) Civilian Personnel Management Mandates 

Personnel management is based on and embodies the Merit System Principles (the 
merit system principles in 5 United States Code 2301(b). The merit system principles are 
the public’s expectations of a system that is efficient, effective, fair, open to ah, free from 
political interference, and staffed by honest, competent, and dedicated employees. 

The merit system principles are: 

• Recruit qualified individuals from ah segments of society and select and 
advance employees on the basis of merit after fair and open competition 
which assures that ah receive equal opportunity. 

• Treat employees and applicants fairly and equitably, without regard to 
political affiliation, race, color, religion, national origin sex, marital status, 
age, or handicapping condition, and with proper regard for their privacy 
and constitutional rights. 

• Provide equal pay for equal work and recognize excellent performance. 

• Maintain high standards of integrity, conduct, and concern for the public 
interest. 

• Manage employees efficiently and effectively. 

• Retain and separate employees on the basis of their performance. 

• Educate and train employees when it will result in better organizational or 
individual performance. 
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• Protect employees from arbitrary action, personal favoritism, or coercion 
for partisan political purposes. 

(7) ATEC HQ Mandates 

AEC is under the Acquisition Demo which provides significant flexibilities to set 
pay and increase salaries based on performance. However, long-term affordability is a 
concern. To ensure fiscal responsibility, consistency throughout ATEC, and adherence to 
the merit principle of equal pay for equal work, ATEC HQ established control points for 
every position and used to set pay and manage salary progression. 

Per ATEC Memorandum, 20 Dec 2011 subject: Command Civilian Acquisition 
Workforce Personnel Demonstration Project (AcqDemo) Control Point Policy, the PY13 
control points for AEC are shown in Table 8. 


Acq Demo Rating 
Group 

Band 

Control 

Point 

(Base 

Salary) 

Notes 

Director 

NH-04 

129,517 

Top of Band 

Technical Director 

NH-04 

129,517 

Top of Band 

Division Chief 

NH-04 

119,554 


Technical 04 

NH-04 

110,737 


Technical 03 

NH-03 

93,175 

Top of Band 

Technical Editor 

NH-03 

78,733 


Technical Editor 

NH-02 

65,371 

Top of Band 

Technical Editor 

NK-02 

44,176 

Top of Band 

Program Specialist 

NH-03 

78,733 

Supports Directorate 

Program Specialist 

NH-03 

74,354 

Supports Division 

Executive Assistant 

NH-02 

60,795 

Supports AEC 

Director 

Admin Specialist 

NH-02 

54,911 

Supports Tech 
Director/ 

Military Deputy 

Secretary 

NK-03 

49,681 


Secretary 

NK-02 

44,176 

Top of Band 


Table 8. AEC Control Points (from Dellarocco, 2011) 
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In 2011, Commanding General, ATEC through vocal command, directed that 
requirements for ATEC System Team Chairs, who are responsible for the design of 
experiments and integrated evaluation, to complete Eean Six Sigma Black Belt 
certification. 

Commanding General, ATEC directed execution of the ATEC Paperless Office 
Program (APOP). The APOP is a phased approach where the first week required 
employees to determine if they really needed to print the document. During the second 
week, all network printers were taken off-line. During the third week, local printers were 
taken off-line. During PY13, APOP executed twice. 

B. THROUGHPUTS 

1. Tasks 

A throughput component is the tasks of the organization. These are the actual 
basic tasks, jobs, or functions performed by the organization. This factor includes how 
they are formalized, how they vary, and what specification is required. 

DOD Dictionary defines essential task as “A specified or implied task that an 
organization must perform to accomplish the mission that is typically included in the 
mission statement.” AEC’s mission statement as documented on the AEC web-site is: 
“To plan, support, conduct and provide independent evaluations, assessments, and 
experiments in order to provide essential information to decision-makers.” (AEC, 2013) 

Tasks are governed by ATEC Regulation 73-1 and ATEC Pamphlet 73-1. The 
AEC tasks include: Key steps in the traditional ATEC T&E process include: 

a. Conduct the Early Strategy Review to discuss and approve the 
Evaluation Strategy 

b. Develop inputs to the Test and Evaluation Master Plan (TEMP) 

c. Document the evaluation strategy in the System Evaluation Plan (SEP) 

d. Assess the program’s risks and projecting the program’s Effectiveness, 
Suitability and Survivability capabilities and limitations in the OTA 
Milestone Assessment Reports (OMARs). 


36 



e. Conduct Rock Drills to ensure supporting plans are synchronized and 
resources are available 

f Develop OT Test Plans (OT TPs) that detail each data gathering event 

g. Document the system’s Effectiveness, Suitability and Survivability 
capabilities and limitation in OTA Evaluation Reports (OERs) 

h. Prepare safety confirmations and safety releases. 

There is not variation in the tasks, however, the products produced to support the 
task may be tailored. The only metric supporting these tasks are timeliness based on the 
internal milestones captured in the ATEC Decision Support System. 

2. Technology 

The technology factor in the throughput process refers to the workflow in an 
organization and how it can be described. It includes the activities in the workflow, any 
interdependencies among the work units or activities involved in the work flow, and the 
condition of the physical facilities and equipment used by the organization. 

a. Workflow 

The purpose of the ATEC T&E Process as defined in ATEC Regulation 73-1 and 
ATEC Pamphlet 73-1 is to provide essential information to decision makers through 
planning, conducting, and integrating developmental testing, independent operational 
testing, independent evaluation, assessments and experiments. Eigure 8 shows the AEC 
workflow. 
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Figure 8. Interrelated Processes of DOD Acquisition and 
ATEC T&E Process (from Army Test and 
Evaluation Command, 2013, p. 24). 


The purpose of the ATEC T&E Process is to provide essential information to 
decision makers through planning, conducting, and integrating developmental testing, 
independent operational testing, independent evaluation, assessments and experiments. 
Eigures 9 through 12 depict the key activities with inputs and outputs for each phase of 
the acquisition cycle. 
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Inputs 

Key Activities 

Outputs 

MSA 

■ Joint Capabilities 
Document (JCD) 

■ Initial Capabilities 
Document (ICD) 

■ Universal Joint Task List 
(UJTL) and Army 

Universal Task List 
(AUTL) 

■ Form an AST 

■ For non-major programs (ACAT III) and 
those not on the DOD-oversight list, 
determine the level of evaluation. 

■ Conduct mission analysis by considering 
operational capabilities of the system and 
environmental/threat challenges 

■ Develop concept performance objectives 
with the T&E Working Integrated Product 
Team (WIPT) 

■ Develop T&E strategy 

■ T&E Strategy 
• AST Charter 


Figure 9. Evaluation Activities during Materiel Solution Analysis 
(MSA) Phase (from Army Test and Evaluation Command, 

2013, p. 64) 



Inputs 

Key Activities 

Outputs 

TD 

■ UJTL and AUTL 
- ICD 

■ Draft CDD 

■ Operational Mode 
Summary/Mission Profile 
(OMS/MP) Draft Critical 
Operational Issues and 
Criteria (COIC) 

■ Security Classification 
Guide (SCG) 

• Review COIC 

■ Develop MOP and MOE in context of a 
Pattern of Analysis (POA) 

■ Develop DM 

■ Develop experiment design 

■ Coordinate with developmental and 
operational testers, CBTDEV, and MATDEV 

■ Consider use of M&S 

• Develop Data Source Matrix (DSM) 

■ Conduct Early Strategy Review (ESR) 
and/or Concept In Process Review (CIPR) 

■ Develop SEP 

■ Contribute to TEMP 

■ Coordinate and oversee DT 

■ Develop MS B OMAR 

■ Conduct rock drills 

■ Contribute to the development of the TRP 

■ Approved COIC 

■ Draft TEMP 

■ Test and simulation 
execution strategy 

■ SEP 

■ MS B OMAR 

■ Test Resource Plan 
(TRP) 


Eigure 10. Evaluation Activities during Technology Development (TD) 
Phase (from Army Test and Evaluation Command, 2013, 

p. 65) 
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Inputs 

Key Activities 

Outputs 

EaM 

D 

■ UJTLandAUTL 

■ CDD 

- COIC 

■ OMS/MP 

■ TEMP 

■ SEP 

■ APB 

■ System specifications 

■ Draft ISP 

- TRP 

■ Logistics Manpower and 
Personnel Integration 
Demonstration (LMD) 
plan 

■ Military utility 
assessments or 
demonstrations of final 
reports 

■ SCG 

■ Update the SEP 

■ Update the TEMP 

■ Develop an M&S accreditation plan; 
conduct M&S Verification and Validation 
(V&V) as required; and prepare M&S V&V 
and accreditation reports 

■ Coordinate with testers on the OTA TP 

■ Coordinate/oversee the DT 

■ Conduct a Limited User Test (LUT) 

■ Conduct LMD 

■ Conduct OTRR 

■ Conduct OT 

■ Develop MS C OMAR 

■ Prepare an OAR in support of LRIP for 

ACAT 1, II or oversight systems 

■ TEMP 

■ Updated SEP 

■ OTA TP 

■ Accredited M&S 

■ MS C System 
Evaluation Report 
(SER) 

■ DT event level Test 
Report (TR) 

■ LMD report 


Figure 11. Evaluation Activities during Engineering and Manufacturing 
Development (E&MD) Phase (from Army Test and 
Evaluation Command, 2013, p. 66) 



Inputs 

Key Activities 

Outputs 

P&D 

■ UJTLandAUTL 

■ CDD 

- CPD 

■ COIC 

■ OMS/MP 

- TEMP 

■ SEP 

■ APB 

■ System specifications 

■ ISP 

• Military utility 
assessments or 
demonstration final 
reports 

■ SCG 

■ TRP 

■ OTA TP 

■ Update the SEP 

■ Update the TEMP 

■ Verify and validate M&S as required 

■ Coordinate with testers on OTA TP 

■ Prepare T&E materials for OTRR 

■ Coordinate/conduct OT 

■ Develop OTA Evaluation Report (OER) in 
support of FRP Decision Review (DR) 

■ Updated TEMP 

■ Updated SEP 

■ Validated M&S 

■ OTA TP 

■ OT reports 

■ FRP decision 
review OER 


Eigure 12. Evaluation Activities during Production and Deployment 
(P&D) Phase (from Army Test and Evaluation Command, 

2013, p. 66) 
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b. Tasking Process 

AEC directorate structure is discussed in later in the document. ATEC HQ 
Directorates, as the servicing staff, task AEC directorates without going through an 
operations cell. Up to eleven (11) separate mission analyses (one per AEC directorate) 
are completed to determine if task should be executed by AEC. Eigure 13 shows the 
complexity of the AEC tasking process. 




Eigure 13. AEC Tasking Process 


c. Facilities 

As a result of BRAC 2005, ATEC headquarters and the U.S. Army Evaluation 
Center located in Alexandria, Virginia, were directed to relocate to Aberdeen Proving 
Ground (APG) and to consolidate with elements of the command already stationed there 
by Sept. 15, 2011. ATEC and AEC headquarters staff was consolidated within a new 
headquarters building (B2202); however the new building could not accommodate all 
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AEC employees. This resulted in a “geographically dispersed” AEG with three 
directorates (SVED, ILS & ESD) stationed in renovated “rolling pin” barracks; lED in a 
former Euture Combat System building and SED stationed in stationed in temporary 
space (relocatable buildings) after the former DTC HQ building was condemned due to 
black mold. BMDED, serving as the BMDS OTA, is located with their customer at 
Redstone Arsenal, Alabama. Eigure 14 shows the placement of the AEC APG 
directorates and was derived from Google Maps. 



Eigure 14. AEC Locations (APG) 


B2202 is a newly constructed building that includes 141,453 gross square feet of 
administrative, meeting and training space on three levels. The building incorporates 
several specialized features to include Anti-terrorism Eorce Protection, a work-out room, 
central conference and training areas. B2202 is LEED®-Gold certified; LEED®-Gold is 
a green building certification program that recognizes best-in-class building strategies 
and practices (Eoulger-Pratt, 2009). Eigure 15 shows the new ATEC HQ building. 
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Figure 15. ATEC HQ B2202 

AEC management (Directors, Technical Directors and Division Chiefs) are 
assigned private offices. The employee workspace is configured using cubicles. B2200 
has two sizes of cubicles 6’X8’ and 8’X8’ in the “cube farm.” Each cubicle has side 
panels that are 57” in height and has limited storage. There is a “mobile pedestal file with 
cushion top” which serves as storage and guest seating. There is a storage tower that 
provides lockable file drawers and a wardrobe for coat storage. A bookcase is also 
provided that has a lockable drawer. 

Many cubicles are adjacent to the main hallways and common area break rooms. 
Eack of privacy and noise are the two of the biggest complaints regarding the cubicles as 
the 57” side panels do not shield hallway conversations according to co-workers 
(personal communications, November 12, 2013). 

The AEC HQ Office of the Director is known as the “Eortress” (personal 
communication with co-workers, various). Due to anti-terrorism/force protection 
procedures, there is controlled access to the suite of offices. A visitor control officer is 
stationed within the reception area and ensures that visitors are escorted back to the 
offices in a deliberate manner. 

AEC Director’s policy letter #1 focuses on the Open Door Policy. The intention 
of the Open Door policy is to assist conflict resolution within the center. The guidance in 
the letter states that the issue needs to come up through the chain of command. 
Unfortunately, the AEC workforce interprets “open door” policy as a “walk-in at any 
time.” 
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d. Equipment 

B2202 is equipped with the latest approved equipment to include multifunctional 
products compose of copier, scanner, printer and facsimile with the added capability of 
network-based document capture, storage and distribution. This equipment is accessed by 
logging in with a Common Access Card (CAC). The all-in-one functionality requires 
network availability. 

AEC employees are equipped with laptop with docking station and additional 
monitor. This set-up allows employees to take their computers with them while 
supporting and witnessing tests. 

Information technology systems are addressed in the Processes section of the 

paper. 


3. Structure 

In the OSF model, structure refers to basic groupings of activities and people, 
how the activities are combined or departmentalized, and how groupings are integrated. 
Also considered, are the integrating devices used such as hierarchy, task forces, matrix or 
network type of arrangements. 

AEC is an organized U.S. Army activity, comprised of an Army table of 

distribution and allowances (TDA) unit reporting directly to the Commanding General 

(CG), U.S. Army Test and Evaluation Command (ATEC). As a result of BRAC 2005, 

Headquarters U.S. Army Test and Evaluation Command (ATEC), U.S. Army 

Developmental Test Command (DTC), and U.S. Army Evaluation Center (AEC) 

reorganized and consolidated in accordance with the 2005 Base Realignment and Closure 

Eaw, HAS-JCSG-D-05-36: “Realign Park Center Four, a leased installation in 

Alexandria, VA, by relocating and consolidating Army Test and Evaluation Command 

(ATEC) with its subcomponents at Aberdeen Proving Ground (APG), MD.” The three (3) 

headquarters staff consolidated into one HQ ATEC with a G-Staff structure and 

established staff matrix support to AEC. DTC was disestablished and the test 

management function and resources were split between HQ ATEC G-9 and AEC 

directorates. HQ ATEC staff analyzes, assesses, provides staff-management oversight 
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and recommends for decision all activities affecting policy, guidance, developmental 
processes and implementation/execution processes to support the center in meeting its 
mission. 

The AEC organizational structure is shown at Figure 16. AEG is comprised of a 
headquarters and ten subordinate directorates. The Director of AEC is a one star General 
Officer (GO) billet with a civilian Executive Director (Senior Executive Service (SES). 



BMDED- Ballistic Missile Defense Evaluation Directorate 

C2ED - Command and Control Evaluation Directorate 

FED - Fires Evaluation Directorate 

lED - Intelligence Evaluation Directorate 

MAED - Maneuver Air Evaluation Directorate 

MGED - Maneuver Ground Evaluation Directorate 

SED - Sustainment Evaluation Directorate 

ESD - Evaluation Sciences Directorate 

USED - Integrated Logistics Support Evaluation Directorate 

SVED - Survivability Evaluation Directorate 

Figure 16. AEC Organizational Structure (from Army Evaluation 

Center, 2011) 
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AEC directorates are aligned to the Army’s Warfighter Functions (WFF) and the 
core themes of effectiveness, suitability and survivability. Given the size and the scope of 
the Maneuver WFF, AEC organized around two maneuver directorates (air and ground). 
The technical analytical functions of evaluation sciences (RAM, statistical analysis and 
modeling and simulation), integrated logistics support and survivability manage and 
apportion workload effort in a matrix-support arrangement. 

• Ballistic Missile Defense Evaluation Directorate (BMDED)-Army 
operational test and evaluation arm of the Ballistic Missile Defense 
System (BMDS), and lead service member of the BMDS Operational Test 
Agency Team. 

• Command and Control Evaluation Directorate (C2ED)-Army and joint 
command, control, and communications, business information and 
medical information systems. 

• Fires Evaluation Directorate (FED)-Fire Support and Air and Missile 
Defense systems (rockets and missiles, cannons, command and control) 

• Intelligence Evaluation Directorate (lED)-Intelligence-related acquisition 
programs, surveillance and reconnaissance, electronic and information 
warfare covering national, theater, coalition and commercial space. 

• Maneuver Air Evaluation Directorate (MAED)-Aviation systems to 
include aircraft, air traffic control, munitions and air Soldier support 
systems 

• Maneuver Ground Evaluation Directorate (MGED)-Infantry/Soldier 
systems, wheeled and tracked combat platforms, sensors and target 
acquisition systems, battle command systems, combat training simulators 
and lethal and non-lethal weapons/munitions programs. 

• Sustainment Evaluation Directorate (SED) -Sustainment, mobility, 
maneuver support, quartermaster, ordnance, transportation, military 
police, engineer and chemical-biological systems. 

• Evaluation Sciences Directorate (ESD) - Reliability, Availability and 
Maintainability (RAM) system characteristics for major defense 
acquisition programs; statistical analysis. Design of Experiments (DOE) 
and modeling and simulation (M&S) support; co-lead with the Army 
Materiel Systems Analysis Activity (AMSAA) for the Army’s Center for 
Reliability Growth (CRG). 

• Integrated Eogistics Support (lES) Directorate-Eogistics supportability 
(to include MANPRINT) evaluation of a system and its impact on 
suitability, and independent logistics supportability assessments. 
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• Survivability Evaluation Directorate (SVED)-Survivability, ballistic and 
non-ballistic battlefield threats, live-fire evaluations and reports, and 
vulnerability and lethality of Army and designated joint systems. Also 
leads ATEC’s Information Assurance Task Eorce for the Combatant 
Commanders (COCOM). 



Table 9. Number of Authorizations by Directorate per TDA EY13 


Table 9 shows the number of authorizations per directorate as defined in 
the EY13 TDA. AEC allocates authorizations based on workload. Table 10 shows the 
number of systems/programs supported by ACAT for each directorate as documented in 
the ATEC Decision Support System. 


Directorate 

Acquisition Category (ACAT) 

TOTAL 

ID 

lAC 

lAM 

1C 

II 

III 

NA 

BMDED 

4 





3 

2 

9 

C2ED 

21 

5 

18 

2 

7 

29 

76 

158 

FED 

7 


3 

11 

19 

30 

31 

101 

lED 

4 


9 

1 

8 

24 

56 

102 

MAED 

10 



12 

14 

29 

51 

116 

MGED 

17 


4 

39 

25 

85 

96 

266 

SED 

16 


1 

10 

25 

173 

252 

477 


Table 10. Number of Systems/Programs Supported by ACAT for Each 

Directorate. 
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AEC directorates are organized similarly with the program specialist and lead 
secretary reporting to the Director and the junior secretary, technical editor (optional) and 
technical divisions reporting to the Technical Director. Directorate structure is shown in 
Figure 17. 


C -\ 

Director 

^ _ / 


c ->1 

Technical Director 
V___/ 



Divisions 

(3-6( 


Figure 17. AEC Directorate Structure 


AEC is structured with 46 divisions distributed among the 10 directorates. The 
number of divisions per directorate are a minimum of 3 (BMDED, FED, lED, lES) to 
maximum of 6 (C2ED and SVED). Table 11 shows the distribution of authorizations 
among the AEC divisions. 
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Directorate 

Division 

Auth 

BMDED 

EPDDIV1 

6, 

EPDDIV2 i 6 

I/A DIV I 9 

C2ED 

BUS INFO TECH DIV 

14 

INTEGRATION DIV 

13 

MED INFO TECH DIV 

8 

MISSION CMD DIV 

12 

NETWORKS DIV 

24 

TEST MGT DIV 

6 

ESD 

AV.MSL&C21 DIV 

13 

COM BAT SPT DIV 

12 

GRDCBTFIRESPTDIV 

14 

MISSION SPT DIV 

6 

STATISTICS/DOE DIV 

3 

FIRES 

ADA DIV 

20 

FA DIV 

20 

TEST AND SAFETY DIV 

11 

lED 

AISR DIV 

9 

ECM DIV 

12 

GISRDIV 

9 

INTEL&INFO WARFARE 

3 

OPS & MIL EVALDIV 

8 

TEST PLAN DIV 

8 

ILS 

AIR DIV 

11 

FORCE PROTECT DIV 

11 

GROUND DIV 

12 

MAED 

1 

AH&TIDIV 

8 

MANN ED/UNMANNED SYS 

9 

SCOUT UTIL & SPT SYS 

10 

TEST MGT/SYS SAFE DIV 

6 

MGED 

ADVCONCEPTSDIV 

13 

INFANTRY DIV 

13 

MTD MANEUVER SYS DIV 

13 

MUNITIONS DIV 

7 

TEST DIV 

14 

SED 

CHEMBIODEFDIV 

11 

DEVELOPTESTDIV 

15 

FORCE SPT DIV 

15 

MANEUVERSPTDIV 

13 

TACTICAL MOB DIV 

14 

SVED 

BALLISTAIRDIV 

7 

BALLISTGRDDIV 

11 

COCOM DIV 

16 

MILITARY DIV 

3 

NON-BALLISTICAIRDIV 

15 

NON-BALLISTICGRDDIV 

10 


Table 11. Number of Authorizations by Division by Directorate 
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Current Department of Defense guidance on supervisory to employee ratio is 1:14 
as posted on http://cpol.army.mil/library/permiss/310.html, the United States Army 
Civilian Personnel website. Thirty-nine out of 46 divisions are under the 1:14 guidance. 

4. People 

The OSF model design factor, people, describes the number and types of 
personnel in the organization, including their expectations, motivations, and mindsets, as 
well as their knowledge, skill sets, and abilities. This data assists in analyzing the 
organization and any intended or unintended consequences that may occur when inputs 
are being processed into results. 

During FY12, CG ATEC directed commanders and senior leaders to evaluate 
each and every position and their business processes to ensure only critical vacancies 
were filled. All civilian hiring actions were frozen until each commander completed their 
analyses and was approved by CG ATEC. While the analysis was on-going, CG ATEC 
proposed to reduce ATEC’s current civilian authorizations by 220 beginning in EY14. 
The intent was to reduce civilian strength in anticipation of the Army’s manpower 
reductions and not become a bill-payer in future manpower reductions. AEC’s 
apportionment of the reduction was 32 authorizations. Unfortunately, Army is further 
reducing ATEC manpower. 

AEC’s manpower trend is shown in Eigure 18. The AEC civilian authorizations 
are projected to decrease by 20% from EY13 to EY20 as posted on 
https://fmsweb.army.mil/protected/WebTAADS/Erame_DocTypes.asp, the Army’s Eorce 
Management System website). Eigure 18 shows the AEC growth during the conflict years 
as well as the reductions in the out-years. 
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'02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 

Figure 18. AEC Civilian Manpower Trend FY02-FY18 


Military Manpower Trend. AEC military authorizations are also on a downward 
trend resulting in the lowest number of military authorizations to date. The AEC military 
authorizations are projected to decrease by 10% from EY13 to EY20 as posted on 
https://fmsweb.army.mil/protectedAVebTAADS/Erame_DocTypes.asp, the Army’s Eorce 
Management System website) Eigure 19 shows the decreasing trend of AEC military 
authorizations. 
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Figure 19. AEC Military Manpower Trend FY02-FY20 


Focusing on the FY13 to FY20 timeframe, AFC’s total strength (authorizations) 
are decreasing by 100 personnel or 17%. Table 12 and Figure 20 show the full manpower 
trend. 


Auth 

2013 

2014 

2015 

2016 

2017 

2018 

2019 

2020 

CIV 

429 

407 

407 

390 

371 

352 

344 

344 

MIL 

124 

124 

124 

122 

119 

117 

116 

112 

Total 

553 

531 

531 

512 

490 

469 

460 

456 

From 553 (FY13) to 456 (FY20) = 17% reduction 


Table 12. Total Authorizations FY13-FY20 
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Figure 20. AEC Total Authorization Trend (Military and Civilian) 

a. Demographics 

Military-Civilian. ABC’s total workforce for FY13 is 439 civilian authorizations 
and 124 military authorizations. 



124 

CIV 

■ MIL 
439 


Figure 21. Military-Civilian Authorizations 
(1) Military Rank-MOS 

AEC military personnel are comprised from several branches of the military 
shown in Table 13. AEC updates the MOS and rank requirements during the yearly TDA 
update. The requirements are based on the workload projections of weapon systems 
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scheduled for evaluation. Manning AEC with Soldiers can very dynamic. The Army 
assigns officers based on availability. It sometimes takes up to two years to fill a 
requisition for a high demand MOS. 


Branch 

BG 

06 

05 

04 

03 

02 

W5 

W4 

W3 

W2 

E9 

E7 

E6 

Total 

Air Defense 



2 

5 

1 



1 

1 



2 

1 

13 

Adjutant General 










1 


1 


2 

Armor 



1 

1 

1 









3 

Aviation 


1 

1 

2 

1 







3 


8 

Branch Immateriel 


2 




1 





1 



4 

Counter Intelligence 


1 

1 


3 









5 

Chemical 



1 


1 









2 

Field Artillery 


1 

1 

4 

1 

1 








8 

Finance 





1 







2 


3 

General Officer 

1 













1 

Infantry 




2 

1 









3 

Loqisitics 


1 

3 

2 

3 







2 


11 

Military Intelliqenve 


1 

1 

2 

3 







2 


9 

Medical Services 





1 









1 

Ordnance 







1 


1 





2 

Operations Research (FA 49) 




4 










4 

Ouarterm aster 








1 






1 

Acquisition (FA 51) 



9 

17 

3 









29 

Siqnal Corps 


1 

2 

6 





1 



4 


14 

Information Sys Enqr (FA 24) 





1 









1 

Total 

1 

8 

22 

45 

21 

2 

1 

2 

3 

1 

1 

16 

1 

124 


Table 13. AEC Military-Branch by Rank Totals 


(2) Civilian Series-Grade 

AEC civilian personnel are comprised from several branches of the military 
shown in Table 14. AEC updates the MOS and rank requirements during the yearly TDA 
update. The requirements are based on the workload projections of weapon systems 
scheduled for evaluation. 
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Series 

Grade 

SES 

02 

03 

04 

Total 

00180- Engineering Psychologist 



1 


1 

00301 - Misc Admin and Program 


4 

14 

1 


00303 - Misc Clerk and Assistant 


1 



1 

00318-Secretary 


14 



14 

00343- Management and Program Analysis 



2 


2 

00344-Management and Program Clerical Assistant 


1 



1 

00346 - Logistics Management Specialist 



8 

2 

10 

00801 - General Engineer 

1 


40 

29 

70 

00830-Mechanical Engineer 



13 

10 


00850- Electrical Engineer 



1 


1 

00854-Computer Engineer 



4 


4 

00855- Electronics Engineer 



28 

16 

44 

00861 - Aerospace Engineer 




1 

1 

01083-Technical Editor 


3 

1 


4 

01087 - Editorial Assistant 


1 



1 

01301 - Physical Scientist 

1 


1 


2 

01515 - Operations Research Analyst 



120 

112 


01529-Math-Stat 



4 

2 

6 

02210- IT Specialist 



3 


3 

Total 

2 

24 

240 

173 

439 


Table 14. AEC Civilian by Job Series and Grade 


b. Motivations 


Herzberg’s Motivation-Hygiene Theory separates factors into two distinct 
categories that can affect employee morale. One set of factors can promote job 
satisfaction and includes recognition for achievement, advancement, and possibility of 
occupational growth. Completely different factors are more responsible for job 
dissatisfaction including how employees are supervised, policies, co-worker relationships 
working conditions and salary (Hertzberg’s Motivation-Hygeine Theory (Two Factor 
Theory), n.d). 

Furloughs, elimination of monetary awards, restrictions on performance pay-outs, 
a moratorium on overtime/compensatory time and program cancellations such as the 
Student Foan Repayment Program have led to lower moral. The curtailment of training 
and development opportunities (cancellation of conferences) has eliminated opportunities 
for younger employees to present their work to their peers. 
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AEC uses recognition as a motivator. The goal of the awards program is to foster 
mission accomplishments by recognizing professional excellence and motivate the 
workforce to high levels of performance and service. The awards are honorary; monetary 
awards are linked to the CCAS pay-outs. AEC awards program is discussed later in the 
paper. 

Other motivators include benefits such as telework and flexible/compressed work 
schedules. Telework is a benefit of federal employment that allows work from home or 
another convenient location. The Telework Enhancement Act of 2010 resulted in each 
agency must establish a telework policy, so eligible employees can have the opportunity 
to utilize this advantage. The Eederal Employees Elexible and Compressed Work 
Schedules Act (“EEECWA”) authorizes alternative work schedules (AWS): flexible work 
schedules (EWS) and compressed work schedules (CWS). AEC has established core 
hours when all employees must be at work (0900-1500) and allows employees to choose 
arrival and departure times around those core hours. Under a CWS, an employee’s bi¬ 
weekly, 80-hour work requirement is scheduled by the agency for less than 10 days. Most 
AEC civilian employees take advantage of the CWS by working 9 hour days Monday- 
Thursday with one 8-hour Eriday. 

5. Processes/Subsystems 

The last factor in the throughputs portion of the OSE model are the essential 
processes or subsystems in the organization that help manage, control, and run daily 
operations and plan for effective and efficient long term success. The OSE breaks down 
the processes into sub-factors of financial management, measurement and controls; 
human resources management; and communications, information planning and decision¬ 
making. 

a. Financial Management, Measurement and Controls 

The majority of financial management for AEC is handled by its higher 
headquarters. 
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(2) Resource Accountability 

Program Budget Advisory Committee (PBAC). The purpose of the PBAC is to 
provide recommendations to the AEC Director on the administration and management of 
AEC financial and manpower resources. Currently, the PBAC schedule is not time-based 
but event-based. Members include the AEC Military Deputy and AEC Directors. ATEC 
HQ G-8 Director of Resource Management provides the information to the attendees. 

(3) Budgeting 

The majority of financial management for AEC is handled by its higher 
headquarters. Theoretically, the budget is based on the number of authorized positions in 
the TDA along with a certain amount per person for basic supplies, training and travel 
needs. ATEC G-8 uses the estimate of $149K for civilians (includes salary and benefits 
paid for by AEC). AEC is not resourced for 439 civilians. There is disconnect between 
the manpower authorized and the budget allocated for civilian salaries causing confusion 
in out-year workforce planning. 

(4) Control 

Management Internal Control Program. The Accounting and Auditing Procedures 
Act of 1950, as amended by the Eederal Managers’ Einancial Integrity Act (EMEIA) of 
1982, P.E. 97-255 (codified at Title 31, United States Code, Section 3512 (31 USC 
3512)) established the requirement for Government agencies to institute and maintain 
adequate systems of internal control. Office of Management and Budget (0MB) Circular 
A-123, and within DOD by DODI 5010.40 implements the statute. Internal controls are 
defined as “The rules, procedures, techniques, and devices employed by managers to 
ensure that what should occur in their daily operations does occur on a continuing basis. 
Internal controls include such things as the organizational structure itself (designating 
specific responsibilities and accountability), formally defined procedures (for example, 
required certifications and reconciliations), checks and balances (for example, separation 
of duties), recurring reports and management reviews, supervisory monitoring, physical 
devices (for example, locks and fences), and a broad array of measures used by managers 
to provide reasonable assurance that their subordinates are performing as intended” (U.S. 
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Army, 2012). AEC provides a self-assessment using the checklists provided in the DOD 
policies, Army Regulations and local policies to assess if AEC is compliant with the 
guidance. ATEC HQ includes AEC in their annual assessment for the functions provided 
by the staff. 

Monthly Resource and Analysis (R&A) Meetings. R&A meetings are scheduled 
once a month with the Commanding General, ATEC. The purpose of the R&A is to 
provide staff updates and review the command metrics. A separate R&A to the AEC 
Director is not offered. 

(5) Performance Measurement 

a. Civilian Appraisal System 

The National Defense Authorization Act (NDAA) for Eiscal Year (EY) 1996, as 
amended by section 845 of the NDAA for EY 1998, allowed the Department of Defense 
(DOD), with approval of the Office of Personnel Management (0PM), to conduct a 
personnel demonstration project with its civilian acquisition workforce. The DOD 
Civilian Acquisition Workforce Personnel Demonstration Project (AcqDemo) was 
implemented on Eebruary 7, 1999, in accordance with the Eederal Register notice (64 PR 
1426), January 8, 1999. AcqDemo was an opportunity to re-engineer the civilian 
personnel system to meet the needs of the Acquisition, Technology, and Eogistics 
(AT&E) Workforce and to facilitate the fulfillment of the DOD acquisition mission. 

The purpose of AcqDemo was to demonstrate that the effectiveness of the AT&E 
Workforce could be further enhanced by allowing greater direct managerial control over 
personnel functions and, at the same time, expanding the opportunities available to 
employees through a more responsive and flexible personnel system. It was designed to 
provide managers the authority, control, and flexibility needed to better manage the 
AT&E Workforce, with the immediate goal of enhancing the quality and professionalism 
of that workforce, and the ultimate goal of providing the best acquisition systems for the 
DOD. 
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The AcqDemo Project implemented civilian personnel system changes, or 
interventions, designed to overcome the limitations of the existing Title 5 personnel 
system. Together, the AcqDemo interventions: 

• Delegated and streamlined position classification and assignment 
processes; 

• Gave managers a wider range of applicants and the ability to set pay; 

• Linked pay and awards to employee contribution to mission; and 

• Rewarded high contributors, and encouraged low contributors to improve. 

Similarly, AcqDemo was designed to provide the following opportunities to 

employees: 

• Allow rapid advancement without cumbersome promotion procedures; 

• Provide flexibility to adequately compensate (salaries and awards) 
employees; 

• Link employee work assignments to the mission of the organization; and 

• Expand opportunities for training and development. 

AEG employs the Contribution-based Compensation and Appraisal System 
(GCAS) for civilian performance management under the AcqDemo. GCAS is the set of 
appraisal processes that measure an employee’s contribution to the mission of the 
organization. It replaces the Title 5 Civil Service General Schedule (GS) classification 
and pay system. 

AEG employees are annually rated against six factors under GCAS: problem 
solving, teamwork and cooperation, customer relations, leadership and supervision, 
communication, and resource management. Descriptors and discriminators specific to the 
three career paths and broad band levels serve as the rubric by which ratings are 
determined. Salary adjustment and award decisions are linked to, and based on employee 
contribution. GCAS provides a great deal of compensation flexibility: employees have 
the opportunity to earn a larger-than-average salary increase if their contribution justifies 
it. On the other hand, employees could earn a smaller increase, or no increase at all 
(except locality pay), if their contribution does not measure up. Under GCAS, employees 
are rewarded just as their contributions merit; those not contributing at the required level 
will have to do better in order to receive monetary rewards. 
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Under CCAS, broadband classification and pay system replaces the General 
Schedule (GS) structure. The 15 grades of the General Schedule were used to classify 
positions and, therefore, to set pay. The salary range for a given broadband level of a 
career path corresponds to the General Schedule rates of basic pay for the grades that a 
particular broadband level encompasses. 

Occupations with similar characteristics are grouped together into three career 
paths. Each career path consists of a number of broadband levels representing the phases 
of career progression that are typical for the respective career path. The broadband levels 
within each career path are shown in Figure 22, along with their GS equivalents. 
Movement within the broadband levels is based upon contribution. Movement to a higher 
broadband level is a competitive action (Department of Defense, n.d.). 


Business Management and Technical Management Professional (NH) 


I 

II 

III 

IV 


(GS 1-4) (GS5-11) (GS 12-13) (GS 14-15) 

Technical Management Support (NJ) 


I 

II 

III 

IV 

(GS 1-4) 

(GS 5-8) 

(GS 9-11) 

(GS 12-13) 


Administrative Support (NK) 


I 


II 


III 


(GS 1-4) 


(GS 5-7) (GS 8-10) 


Figure 22. Broadbands and Paybands (from Department of Defense, 

n.d.) 


As indicated in Figure 22, there are four broadband levels in the demonstration 
project, (labeled I, II, III, and IV) for two career paths: Business Management and 
Technical Management Professional, which encompasses GS-1 through -15; and 
Technical Management Support, which includes GS-1 through -13. There are three 
broadband levels for the Administrative Support career path, which includes GS-1 
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through -10. Comparison to the GS grades’ rates of basic pay is used in setting the upper 
and lower dollar limits of the broadband levels. Once the employees are moved into the 
demonstration project, GS grades no longer apply. 0PM classification standards are used 
to identify proper series and occupational titles. 

Table 15 shows AEG TDA authorizations by career path and broadband. AEG 
total civilian authorized strength is 439. Table 15 does not include the 2 Senior 
Executives. 


Broadband 

Paybanc 


02 

03 

04 

Total 

NH- Business Management andTecluucal Management 

8 

240 

173 

421 

N.T-Tecluucal Management Support 

0 

0 

0 

0 

NK - Admimstrative Support 

16 

0 

0 

16 

Total 

24 

240 

173 

437 


Table 15. AEG Civilian Personnel by Broadband and Payband 

b. Military Evaluation Reporting 

The AEG military use the Army Evaluation Reporting System (ERS). The ERS 
evaluates the performance and potential of officers and noncommissioned officers 
(NCOs) as documented by Army Regulation 623-3 Evaluation Reporting System. The 
basic purpose of the ERS is to provide information for decisions on promotion, retention, 
and assignment, and to provide feedback to the individual. The evaluation report provides a 
centralized selection board the information necessary to individually rank order a population 
under consideration for promotion, retention or separation. 

b. Human Resource Management 

Personnel management is an essential, valuable, and expensive resource to 
manage for the organization. HR processes and procedures outlined in federal, DOD, and 
Department of the Army (DA) personnel rules and regulations assist in management of 
this resource. AEG is serviced by ATEC G-1 Directorate of Human Resources who is 
responsible for the processing of personnel actions to include recruitment. The staffing 
strategy and planning remain with AEG management. 
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(1). Recruitment, selection, retention, termination and/or retirement 

The Office of Personnel Management and the Army Civilian Personnel system 
rules, procedures, and regulations govern the recruitment of AEC civilian personnel. 
Selection of new hires and promotions are primarily achieved through the same 
competitive process. All qualified applicants apply via an on-line civilian personnel 
system USAJOBS. Job announcements are posted for a specified period of time. 
Applicants meeting the predetermined qualifications (education requirements as well as 
Army Acquisition Corps certification) are referred from CPAC to the selecting official at 
AEC. Resumes are reviewed and placed into a competitive range to conduct interviews. 
The hiring official facilitates an interview panel which consists of subject matter experts 
and a representative of ATEC HQ when hiring a senior employee. Interviews are 
conducted and a selection is made. The entire process is reviewed by the ATEC Equal 
Opportunity Office. 

Due to financial constraints, the civilian hiring process is subject to additional 
scrutiny to include re-reviews of positions previously approved for hiring, additional 
Civilian Personnel Advisory Center validation of hiring and selection approvals (to 
include incumbents previously validated for qualifications), and the initial area of 
consideration being limited to Army candidates in the local commuting area. 

AEC also employs the Department of the Army (DA)’s Career Intern Program. 
The DA interns enter the program at the GS-5 and GS-7 levels as permanent full-time 
employees. Interns receive career/career-conditional appointments in the competitive 
service. DA interns reside on HQDA student detachment spaces and are funded by 
HQDA for the first 24 months. Upon graduation from the program, interns are placed on 
mission rolls in journey level GS-9 or GS-11 positions, according to the career program 
intern target grade and availability of placement positions. 

AEC faces a growing challenge with many key employees becoming eligible to 
retire within the next five years. Of the 372 civilian full-time employees working within 
the organization in EY13, 10% are eligible for immediate regular retirement and another 
24% are eligible for early retirement (Eigure 23). The regular retirement eligible 
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population will grow significantly over the next five years to 20%. While AEC has not 
experienced a full out “wave” of retirement eligible employees during the past five years, 
succession planning and strategic goals must reflect the preparedness for such an 
occurrence. 


AEC Retirement Profile 



Figure 23. AEC Retirement Profile 

ATEC is currently facing a turnover rate of 7% annually and anticipates this 
percentage to continue. 


AEC Separation Profile 



Figure 24. AEC Separation Profile 
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(2) Position Descriptions 

A position description (PD) is a statement of the major duties, responsibilities, 
and supervisory relationships of a position. In its simplest form, a PD indicates the work 
to be performed by the position. The purpose of a PD is to document the major duties and 
responsibilities of a position, not to spell out in detail every possible activity during the 
work day. 

AEC employees were under CCAS, NSPS and transitioned back to CCAS. From 
2005-2011 timeframe, AEC was relocating from Alexandria to Aberdeen Proving 
Ground and supporting rapid acquisition. AEC employed PDs that provided great 
flexibility in hiring talent. The positions were designated as interdisciplinary (many job 
series assigned to a position) to allow for many selections against one job announcement. 
PD 90979 NH-****-03 Evaluator was open to 20 different job series in the mathematics, 
engineering and physical science disciplines. The major duties of PD 90979 read: 

Serves as a lead or member of an interdisciplinary team responsible for the 
planning, execution and reporting of a comprehensive evaluation of the 
effectiveness, suitability, and survivability of weapon systems in the 
acquisition process. The positions to be filled cross all mission areas and 
include one or more of the following areas of expertise: Technical 
performance of weapon systems, reliability analyses. Integrated Eogistics 
analyses, all areas of Survivability and lethality analyses. Modeling and 
Simulation, database development. Statistical analysis. Operations 
Research Analysis, Software analysis, and analysis of the Operational 
capabilities and limitations of weapon systems. “Weapon Systems” 
include all Army materiel in the Acquisition Process, systems to be fielded 
to Army units or Joint systems in which the Army has significant 
participation. Eeads or contributes to the development of all Test and 
Evaluation related documentation in ATEC including the System 
Evaluation Plan, the Individual Event Plan, System Assessments, the 
System Evaluation Report and the System Analysis Report. Reviews or 
contributes to all T&E related documentation prepared by other major 
commands to include Test and Evaluation Master Plans, Operational 
Requirements Documents, and Critical Operational Issues. Monitors test 
execution, data collection, and data base development. Synthesize data 
from, modeling and simulation, experimentation, technical and operational 
testing to assess overall system capabilities and limitations. Interacts with 
program managers and other members of the acquisition community to 
ensure a comprehensive Test & Evaluation program is conducted. 
Prepares written analysis, evaluations, and briefings to support Army and 
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DOD materiel decision-making. Develops, presents, and defends 
presentations within ATEC, to senior Army leadership, DOD officials, and 
at appropriate symposia. 

The PD flexibility for hiring does not lend itself to workforce reshape actions 
such as Voluntary Early Retirement Authority (VERA)/Volunteer Separation Incentive 
Program (VSIP) as it appears as if all individuals on the same PD have the same exact 
skill mix.. The commodities that AEG evaluates differ and requires different subject 
matter expertise (air defense systems have different technological and operational 
capabilities than C4ISR systems). 

(3). Right People on Board 

The manning document or “working TDA” is the tool that tracks “faces” to TDA 
“spaces.” The manning document is used to track the on-board workforce to include 
identifying vacancies, employees on detail or temporary promotion, local interns and the 
Internship Program (formerly known as the Student Career Experience Program (SCEP) 
and Student Temporary Employment Program (STEP). 

The Office of Personnel Management (0PM) developed position classification 
standards to define occupation series, establish official position titles and describes the 
various levels of work. Additionally, 0PM developed the General Schedule Qualification 
standards which an individual must meet to be hired into the position (Office of 
Personnel Management, n.d) http://www.opm.gov/policy-data-oversight/classification- 
qualifications/general-schedule-qualification-standards/#url=Group-Standards. Table 16 
shows the current on-board occupational series by grade. 
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Series 

Grade 

SES 

02 

03 

04 

GS- 

11 

Total 

00180- Engineering Psychologist 



1 



1 

00301 - Misc Admin and Program 


2 

13 



15 

00318- Secretary 


3 

9 



12 

00344-Management and Program Clerical Assistant 


2 




2 

00346- Logistics Management Specialist 



10 

3 

1 

14 

00401 - Natural Sciences - Biology 



4 



4 

00801 - General Engineer 

1 


23 

34 

1 

59 

00830-Mechanical Engineer 



29 

25 

3 

57 

00850- Electrical Engineer 



11 

5 


16 

00854-Computer Engineer 



4 

1 


5 

00855- Electronics Engineer 



6 

14 


20 

00861 - Aerospace Engineer 



3 

1 


4 

00893-Chemical Engineer 



4 

1 


5 

01083-Technical Editor 


1 

2 



3 

01087 - Editorial Assistant 


1 




1 

01301 - Physical Scientist 

1 


5 

3 


9 

01320-Chemist 



3 

“n 


4 

01515 - Operations Research Analyst 



61 

60 


121 

01520 - Mathem atician 



6 



6 

01529-Math-Stat 



7 

1 


8 

01550 - Computer Scientist 



3 



3 

02210- IT Specialist 



4 

3 


7 

Total 

2 

9j 

208 

152 

5 

376 


Table 16. AEC Civilian On-board by Occupational Series and Grade 


Most of AEC’s position are classified as professional and scientific positions and 
require a bachelor’s or higher degree. However, there are a few series where the degree 
requirement is not aligned with AEC needs. An example is the 1515 Operations Research 
Analyst. The 0PM general schedule qualification standard for Operations Research 
series, 1515 as published on the 0PM website states: 

Degree: in operations research; or at least 24 semester hours in a 
combination of operations research, mathematics, probability, statistics, 
mathematical logic, science, or subject-matter courses requiring 
substantial competence in college-level mathematics or statistics. At least 
3 of the 24 semester hours must have been in calculus. 

Courses acceptable for qualifying for operations research positions may 
have been taken in departments other than Operations Research, e.g.. 
Engineering (usually Industrial Engineering), Science, Economics, 
Mathematics, Statistics, or Management Science. 
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Degrees in economics, management science and business do not necessarily offer 
the level of probability and statistics and high-level mathematics required to support the 
AEC mission. 

(4). Civilian Age 

AEC has a mature workforce; average age is 45, median age is 47. Youngest 
employee is 19 years of age and oldest employee at 71 years of age. 
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_Q 

Table 17. Civilian Age Profile 
(5). Civilian Eength of Service 

Although AEC’s workforce is fairly mature, the average years of service is 13 
years. Eive employees have less than a year of service; one employee has over 40 years 
of service. The low average years of service may be due to lack of military time or 
contractor time counted. 
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Total 


Table 18. Civilian Years of Service 


(6). Training and Development 

Training programs are developed specifically to enhance technical, administrative 
or procedural understanding of ABC’s mission. ABC’s mission depends on the 
development of the workforce’s technical and leadership skills. 

a. Civilian Bducation System 

As a result of changing roles and responsibilities of the Army Civilian Corps, the 
Army implemented the Civilian Bducation System (CBS) in 2007, modeled after the 
established officer and NCO education system. The CBS provides progressive, sequential 
leader development training and education. Army Regulation 350-1 (2009) states the 
CBS will “prepare agile and innovative Army civilians who can lead during times of 
change and uncertainty; are prepared for the rigors of service as multi-skilled leaders; and 
are armed with the values, skills and mindset to serve as competent, resilient supervisors 
and managers.” Courses include the Boundation Course, Action Officer Development 
Course, Supervisor Development Course, Basic Course, Intermediate Course, and 
Advanced Course. All ABC employees are required to complete the CBS program in 
order to be considered for senior level positions. 
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Overview of the Civilian Leader 
Development Program 


Pay Band Eauivalent 

Pay Band Eauivalent 

Pay Band Eauivalent 

GS-5f7/9 1 GS-11 

GS-12 I 'GS-13 

GsT4“ GS^15“ 


Leader development training, education . 
experience 

-Civilian Education System (CES) is the underpinning 
of all leader development programs 

- Courses target employees at their GS 
grodes/equivalcnt pay txind 

Aligned with DOD Leader Competencies 


OaftnM Senior Loader 
Development Program (DSLDP) 

Senior Service College 

Continuing Education for 
Senior Leadera (CESL) 


Advanced Course (AC) - DL & Resident 
DOD Executive Leadership Program (DELDP) 

Inteimediate Course (1C) - DL & Resident 

OOOCMNan Emerglr>g Leader 


Basic Course |BC1 - DL & Resident 
Manager Development Course (MDC) - DL 
Supervisor Development Course (SDC) - DL 
Action Officer Development Course (AODC) - DL 

Foundation Course (FC) - DL For ALL new Army civilians 


CES Courses Online Courses DoD Leader Development Programs 

Figure 25. CES Leader Development Program (from U.S. Army, n.d.) 


b. Career Program Training. 

Career paths blend the leadership, management, scientific, and functional 
competencies, assignments, and training guidance needed by civilians who aspire to key 
civilian leadership positions within the Army in specific career programs. 

AEC civilian career programs (CPs) as documented on the approved FY13 TDA: 
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CP 

Number of 
Authorizations 

Description 

16 

97 

Functions as research, design, development, test and evaluation, 
production, installation, operation, and maintenance; data collection; 
processing and analysis; and material resources operations. 

17 

10 

Materiel Maintenance Management; Maintenance Management starts 
with the initial concept and design of materiel and follows through 
sustainment and life cycle extension; concerned with the reliability and 
maintainability of the new item, its technical description, and the 
supporting publications which describe how to properly use and maintain 
it. 

34 

3 

Information Technology Management; support the Army’s critical 
IT/Cyber mission; 2210 Series (IT Management) consists of the 
following eleven parenthetical titles: Application Software; Customer 
Service; Data Management; Enterprise Architecture; Internet; Network 
Services; Operating Systems; Policy & Planning; Security; Systems 
Administration; and Systems Analysis. The 391 Series 
(Telecommunications) and 30Ti Series (Information Management) 

36 

155 

Analysis, Modeling and Simulation; use of simulation to improve 
training, mission, rehearsal, planning, experimentation, acquisition, and 
operations. 

BLANK 

174 



Figure 26. AEC Civilian Personnel by Army Career Path 


39.6% of AEC authorizations show no career path selected. As a result, the 
incumbents in these positions may not receive supported training and education plan from 
centralized funding. This places a burden on an already stressed AEC budget. 

c. Acquisition Corps Requirements. 

The Defense Acquisition Workforce Improvement Act (DAWIA) of 1990 
mandated the establishment of an Acquisition Corps in each of the Services and at least 
one corps for DOD agencies. The purpose of the Acquisition Corps is to certify and 
recognized the acquisition workforce for having achieved professional status. 
Certification is the procedure through which a military service or DOD Component 
determines that an employee meets the education, training, and experience standards 
required for a career level in any acquisition, technology, and logistics career field. 
Civilian positions and military billets that are in the DOD acquisition system, have 
acquisition duties, and fall in an acquisition position category established by the Under 
Secretary of Defense for Acquisition and Technology (USD(A&T)). There are several 
acquisition position categories to include auditing; business-cost estimation; business- 


70 





financial management; contracting; engineering; facilities engineering; information 
technology; life cycle logistics; production, quality and manufacturing; program 
management; purchasing; science and technology management; and test and evaluation. 

AEC’s positions are coded as follows: 


T-Test and Evaluation 

352 

A-Program Management 

82 

E-Eife Cycle Eogistics 

27 

S - SPRDE-Systems Engineer 

2 

Z - Unknown 

8 

2-Acquisition Attorney 

1 

Blank 

91 


563 


Figure 27. AEG Civilian and Military Acquisition-coded Positions 


The position coded as an Acquisition Attorney is in error (position is Test and 
Evaluation). The 8 positions coded with ‘Z’ should be ‘A’ (Program Management). The 
91 positions with no acquisition coding include program specialists, executive assistants, 
technical editors, secretaries, COCOM lATF Division (non-acquisition mission) and 
operational officers and NCOs. 

d. AST 101. 

AST 101 focuses on the roles and responsibilities of the ATEC System Team 
(AST), the technical and project management skills necessary to be an evaluator or 
analyst, references and resources for job assistance, and the opportunity to complete 
practical exercises relevant to the AST function (Boggs, 2012). 

e. Test and Evaluation Basic Course. 

The Test and Evaluation Basic Course (TEBC) consists of two parts, an on-line 
training (GET) and a Resident Seminar (RS). Both are essential components of ATEC’s 
initial training program. Personnel in Test and Evaluation positions must complete the 
GET within the first 2 months of assignment and must also complete a Resident Seminar 
(RS) within 6 months. Personnel in positions that are not part of an ATEC System Team 
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are not required to take the TEBC RS. The TEBC OET provides a mandatory in-depth 
orientation for all newly assigned personnel. The TEBC RS is a detailed (35 hours) 
seminar that provides “how-to” instruction by subject matter experts to ATEC testers, 
evaluators, analysts and other personnel as designated by their chain-of-command. 

f. New Employee Orientation. 

In concert with HQ ATEC, AEC executes a new employee orientation (NEO) for 
its employees (military and civilian) every six months. The purpose of the NEO is to 
provide training on the mission, tasks, structure, processes and procedures of the 
organization. Day 1 consists of command overviews are provided of each ATEC element. 
Each staff element provides an overview of services provided as well as points of contact. 
A working lunch is scheduled which provides an opportunity for employees to ask 
questions of the ATEC Command Group. The NEO ends with a tour of Aberdeen Test 
Center which provides a first-hand look at the test capabilities available. Day 2 consists 
of briefings focusing on AEC to include overviews of the AEC directorates; question and 
answer session with the AEC Command Group; CSM Top Nine; AST Basics & Mock 
AST; Program Manager’s Perspective; DOT& 101 and DASD (DT&E). Day 3 focuses 
on personnel processes such as rating and awards, training and development, 
administrative tools, Eamily Readiness Group and Army 101 for those civilians that are 
new to the Army. 

g. Career Management Road Maps. 

AEC developed structure, detailed roadmaps for career management. These 
roadmaps include technical training and leadership development and integrate 
requirements from DAWIA, CES and other initiatives. Eormal education, on-the-job 
training and experience play a pivotal role in developing the AEC workforce. Career 
Programs 13/17/24 (Supply, Maintenance & Transportation), 16 (Engineer & Scientist) 
and 36 (Analysis, Modeling and Simulation) are completed and are required for use in 
career counseling and IDP development. 
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h. Developmental Assignments. 

AEC uses developmental assignments to broaden employee capabilities and 
knowledge by providing an opportunity to perform duties in other occupations, functions, 
or agencies. Participants are given broadening experiences in diverse fields through 
various job rotations and cross functional assignments. Employees gain competencies 
necessary to be competitive for positions of greater responsibility, as well as managerial 
and leadership positions within the Department of Defense. At the end of EY13, 16 
individuals were on developmental assignment to Army Materiel Systems Analysis 
Activity (AMSAA), U.S. Army Deputy Chief of Staff G-4 Eogistics, 21st Theater 
Sustainment Command (TSC) Germany, Aberdeen Test Center, ATEC HQ Office of 
Chief Counsel, Edgewood Chemical-Biological Center, DOT&E, Missile Defense 
Agency, Army Research Eaboratory, ATEC HQ G-1 Human Resources Directorate and 
Corps of Engineers. 

i. Mandatory Training 

All ATEC military personnel and Government civilians are required to complete 
mandatory training. Required courses are announced through the ATEC Training Tracker 
Module (ATTM) system. 

• Anti-terrorism, Eevel I 

• Army Substance Abuse Program 

• Army Suicide Prevention Program 

• Suicide Prevention - Pace to Pace 

• Combating Trafficking in Persons (CTIP) Program 

• Composite Risk Management 

• Constitution Day Training 

• Equal Opportunity Program 

• Ethics 

• Operational Security (OPSEC) 

• Sexual Harassment/Assault Response Prevention (SHARP) 

• SHARP Team Bound 

• Subversion and Espionage Directed Against the U.S. Army (SAEDA) 

73 




• Threat Awareness and Reporting Program (TARP) 

• Defense Travel System (DTS) Basic-DTS Travel Documents 

• Programs & Policies-Travel Policies 

• Programs & Policies-Travel Card Program 

• Annual Security Refresher Training 

• Cyber Awareness Challenge 

• Safe Home Computing 

• Personally Identifiable Information (PII) 

• Portable Electronic Devices & Removable Storage Media 

• Phishing Awareness 

• Army G3 Computer Security Training 

In addition to the Army mandatory training, ATEC HQ requires each individual 
to complete “Release of ATEC Test and Evaluation Data.” 

j. Supervisory Training. Defense Authorization Act of 2010, Section 1113, 
established the requirement for services to provide mandatory training for all new and 
experienced supervisors. New supervisors must complete the initial training within a year 
of their assignment. Experienced supervisors must complete refresher training at least 
once every three years. Topics include: Workforce Planning, Position Management and 
Classification, Hiring, Merit Systems Principles and Prohibited Personnel Practices, 
Onboarding, Performance Management, Training and Development, Recognition, 
Incentives and Awards, Coaching, Counseling and Mentoring, Eeave Administration, 
Workers’ Compensation, Eabor Relations, Supervising a Diverse Workforce, Hostile 
Work Environment, Reasonable Accommodations, Creating an Engaging Work 
Environment, Managing Conflict, Valuing Individual Differences, and Eeading Change. 
ATEC HQ G-1 Human Resources Directorate coordinated workshops with outside 
vendors on various topics to increase supervisor competencies. 

k. Military Deader Development and Training. 

Military are assigned to AEC as a broadening assignment. Soldiers assigned to 

AEC are selected to attend Army schooling to include Advanced Noncommissioned 

Officer Course (ANCOC), Command and General Staff College (CGSC) and School for 
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Command Preparation (pre-command course). Military assigned to AEC attend the New 
Employee Orientation, AST 101 and mandatory annual training. 

(7). Reward Programs 

The final sub-factor in HR Management is the rewards program, opportunities for 
advancement, compensation packages, and recognition. 

AEC actively recognizes civilian employees through honorary awards to include 
AEC Civilian Employee of the Quarter, the Baltimore Eederal Executive Board 
Excellence in Eederal Career awards as well as Achievement Medal for Civilian Service, 
Commander’s Award for Civilian Service and Superior Civilian Service Awards. AEC 
also recognizes military members by issuing awards such as the AEC Military Member 
of the Quarter and Military Outstanding Volunteer Service awards. Military are also 
recognized with permanent change of station (PCS) awards when changing duty stations. 

c. Communication, Information Planning and Decision-Making 

(1) Planning 

The Strategic Initiatives Group (SIG) reports directly to the AEC Technical 
Director and performs integration and synchronization of initiatives. The Current 
Operations (OPS) cell reports to the AEC MIEDEP and is responsible for assessing the 
current situation and day-to-day operations of AEC. The SIG is responsible for future 
operations (EUOPS) planning and assessing for the mid-range time horizon and is 
responsible for strategic planning for the mid to long-range time horizons. The SIG and 
OPS use military decision making process (MDMP) and other techniques when making 
recommendations to the AEC leadership. 

(2) How information is gathered 

Within AEC, there are no formal methods for gathering information. AEC uses 

many IT systems to execute the supporting functions to support the mission. 
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Figure 28. IT Systems used by AEC 


Document Management. A minimum of six different systems are used for 
document management within AEC. 

• VISION Digital Eibrary System (VDES) serves as a digital repository for 
all ATEC T&E documentation, providing a complete record of tests and 
necessary reference materials. 

• Sharepoint is a portal for local applications and document management 

• Aurora is an electronic staffing system that provides an automated 
movement of documents through the review and approval process from 
submission through the AEC Office of the Director to ATEC Headquarters 
levels. It provides the ability to create on-line electronic staffing packages 
and automatically manage the movement of these packages through the 
staffing processes. 

• Network drives are available for document storage. The drives are 
partitioned by directorate. Eile sharing across the center is not currently 
supported. 

• Eocal drives are used for working documents. 
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• Army Knowledge On-line (AKO) is an Army-wide document 
management system. AEG uses AKO to share information with 
organizations outside of the ATEC network. 

• Army Records Information Management System (ARIMS) is an Army¬ 
wide records management system. In conjunction with the DA 
requirements and the implementation of Aurora, all final documents will 
be placed into the Army Records Information Management System 
(ARIMS). ARIMS will serve as the final repository for all Government 
business-related documents. 

Test and Evaluation Management. AEG uses one system and one module to 
manage test and evaluation efforts. 

• The ATEG Decision Support System (ADSS) is ATEG’s tool for 
management of T&E activities. It is a database which includes planned 
and actual milestone dates for systems and individual efforts. It is an entry 
point for customers to submit requests for test services. Separate modules 
exist for continuous evaluation (GE), rapid initiatives (RI), developmental 
testing (DT) and operational testing (OT). ADSS is used to release the test 
directive authorizing the ATEG TGs to work on an effort. 

• The Project Management Module (PMM) is the Microsoft® Project 
interface to ADSS. 

Resource Management. As stated previously, ATEG HQ provides staff services 
for AEG. The systems used solely by the ATEG staff are not listed. 


• General Eunds Enterprise Business System (GEEBS) is the Army’s 
financial, asset and accounting management system. 

• RM Online is a web-based, integrated resource management system for 
budgeting, manpower and personnel 

• EMS Web is the Army system to documents manpower and equipment 
requirements and authorizations 

• Automated Time Attendance & Production System (ATAAPS) is the 
DOD system that accurately record time and attendance while capturing 
labor hours by job order (task). 

Personnel Management. AEG personnel access several systems to manage their 

careers from performance management to professional development. 

• ATEG Training Tracker Module (ATTM) is the system in place to manage 
annual mandatory training. 

• Defense Givilian Personnel Data System (DGPDS) is the database of 
record for Army civilian personnel data 
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• CAS2NET is the performance management system for acquisition 
demonstration employees. 

• Career Acquisition Personnel and Position Management Information 
System (CAPPMIS) is a portal for acquisition personnel to access 
applications such as Acquisition Career Record Brief (ACRE), Individual 
Development Plan (ID), Senior Rater Potential Evaluation (SRPE), 
Certification Management System (CMS), Army Acquisition Professional 
Development System (AAPDS) and Army Acquisition Corps 
Management System (AAC MS). 

• Army Career Tracker (ACT) Army Career Tracker (ACT) is a leadership 
development tool that integrates training and education for civilian and 
military. 

• Evaluation Reporting System (ERS) is the performance management 
system for military personnel. 

With the proliferation of IT systems within the Army, there should be a natural 
progression from information management to knowledge management. Army Eield 
Manual EM 6-01.1 Knowledge Management Operations states: 

Knowledge management provides the means to efficiently share 
knowledge, thus enabling shared understanding and learning within 
organizations. To do this, KM creates, organizes, applies, and transfers 
knowledge and information between authorized people. It seeks to align 
people, processes, and tools—to include information technology—within 
the organization to continuously capture, maintain, and re-use key 
information and lessons learned to help units learn and adapt and improve 
mission performance. KM enhances an organization’s ability to detect and 
remove obstacles to knowledge flow, thereby fostering mission success. 
Because collaboration is the key contributor to KM, it is imperative that 
everyone be involved in the process, from the generating force that trains 
and sustains the Soldier to the operating force, which ensures Soldiers 
survive and thrive every day in every circumstance or location 


AEC is in an embryonic state regarding Knowledge Management. An example 
where AEC is progressing in KM is the Center for Reliability Growth (CRG). CRG is a 
joint AMSAA-AEC partnership that works towards improving reliability by providing 
policy, guidance, standards, methods, tools, and training. The CRG maintains a collection 
of key reliability tools, models, and documents. By capturing and archiving actual test 
metrics/data, the models and tools are validated and improved. 
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(3) How the organization communicates 

It is common knowledge that effective communication at all levels is essential to 
an organization’s success. With the pace of working in a dynamic environment, 
communication increases in importance, but may be inadequate due to time pressures. 
AEC leadership communicates through electronic mail and quarterly “all hands” 
meetings. “Understanding” newsletters with updates on the strategic initiatives, “Ask the 
Director” page, “News you Can Use” are posted to the AEC SharePoint site for personnel 
to review and reference. Regular weekly meetings with the supporting HQ ATEC staff 
are held to maintain consistent communication between the evaluation directors and 
senior management. 

(4) How decisions are made 

ADP 5-0 Operations Process governs how Army staffs plan. Due to the 
environment of BRAC and supporting an Army at war, AEC became victim to the 
“tyranny of the urgent.” “Mini-MDMPs” were executed due to collapsed timeframes and 
many of the steps of MDMP as defined in ADP 5.0 were skipped. Decisions were made 
with the best limited information available without supporting, formal analysis. AEC is 
re-establishing formal planning processes and is transitioning out of reactive to proactive 
operations. 

(5) Acquisition and contracting 

AEC currently uses contractors to augment the current civilian and military 
workforce; the products provided are not the conclusions of the final evaluation product 
but may include data reduction and analysis, data bases (and management) and data 
produced by models and simulations. The efforts performed by contractors are fully 
reimbursed by the customer. AEC civilian and military personnel provide technical and 
administrative oversight and control of all contractor efforts. 

C. RESULTS 

The results component of the OSE model includes the organization’s culture, 
outputs, and outcomes 
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1 . 


Culture 


By applying the OSF model, the throughput or design factors, e.g., tasks, 
technology, structure, people, and processes, were used to describe AEC. The next 
portion of the model involves the culture of the organization. Organizational culture 
describes the values and assumptions shared within an organization. Norms are defined 
as the informal rules and shared expectations that groups establish to regulate the 
behavior of their members (McShane & Glinow, 2009, p. 328). 

a. Military-Civilian 

There is a varying mix of civilian and military personnel in all types of positions 
at AEC. Both types of personnel bring different but important skill sets, and both are 
critical to mission success. The military and civilian workforces do have different 
organizational cultures. 

Military culture is based on the unique tradition, mission, structure and leadership 
of military history. Military culture maintains distinct sub-cultures within the Army 
branches, that have unwritten sets of rules, viewpoints, perspectives and operating 
procedures (Military Cultural Awareness for Hiring Managers, n.d.). Some of the main 
characteristics of the military organizational culture include: 

• Highly structured and authoritarian way of life with a mission-focused, 
goal-oriented approach—both explicit and implied 

• Strict sense of discipline, tending to adhere to rules and regulations 

• Strong work ethic with high regard for physical and mental strength 

• Code of conduct and organizational culture that reflects well-defined and 
strongly supported moral and ethical principles 

• Decisive leadership that expects loyalty of subordinates and allies 

In contrast, the AEC civilian culture is slightly different than the military culture. 
AEC’s civilians are mostly scientists and engineers and were brought up to ask “Why?” 
Unfortunately, the question “Why?” is perceived as a challenge to the order or task given. 
Many times the civilians need to understand the order or task given prior to execution. 
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There are varying rules and expectations for military and civilian personnel within 
AEC, even when holding the same positions and performing the same type of work. An 
example is hair grooming standards. Army Regulation 670-1 Wear and Appearance of 
Army Uniforms and Insignias governs hair and grooming practices for the Soldiers 
assigned to AEC. Soldiers are authorized to leave their duty station and get a haircut; 
civilians have no requirement for hair and grooming (other than hygiene) and therefore, 
haircuts are scheduled for outside of working hours. 

Other difference between the military and civilian workforce culture include: 

• Attire-the military can wear the same uniform to any meeting; civilians 
wear suits to high-level briefings 

• Time & Attendance-military are “24/7”; civilians are 40 hours per week 

• Physical Training-required by military and is authorized as part of the 
work day; optional and is on the civilians’ own time 

• Training Holidays-military can take advantage of a training holiday which 
is a free day that does not count as leave; civilians do not have this option. 

• Eeave-Military earn 30 days of leave a year regardless of time served; 
civilians start with 4 hours per pay period, increasing to 8 hours per pay 
after 15 or more years of service 

• Sick leave-Military do not have sick leave; civilians earn 4 hours per pay 
period 

AEC effectively integrates the two cultures, while allowing both cultures to exist. 
Civilians are invited to the Officer Professional Development (OPD) sessions as well as 
celebrations such as the Army’s Birthday cake cutting. AEC executes several team¬ 
building to include the “Turkey Bowl” which is an officer against NCO flag football 
game with civilians augmenting both teams. CG, ATEC leads monthly runs the last 
Eriday of the month. The run is mandatory for military and civilians are invited and 
encouraged to participate. 

b. Conflict Resolution 

A common obstacle to effective management and building consensus in many 
organizations is the reluctance of personnel to elevate issues to the supervisors and senior 
leaders for resolution. This can result in wasting valuable time while attempting to solve 
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problems that: 1) are beyond the control of action officers, and 2) within senior leaders’ 
ability to resolve easily based on relationship or having a broader view of the multitude 
of challenges the organizations and the Army are facing. Some consider elevation of an 
issue as a weakness (ie. they failed because they could not individually solve the issue). 

Only one formal policy exists for conflict resolution. Director’s policy letter #1 
focuses on the Open Door Policy. The intention of the Open Door policy is to assist 
conflict resolution within the center. The guidance in the letter states that the issue needs 
to come up through the chain of command. Unfortunately, the AEG workforce interprets 
“open door” policy as a “walk-in.” 

2. Outputs 

Outputs of a system are the goods and/or services produced by the organization. It 
is important in the application of the OSF model to recognize how the outputs are 
measured and to identify the indicators of performance. The outputs for AEG take the 
form of information, formal and informal, written and verbal. A listing of AEG developed 
products as defined in ATEG Regulation 73-1 to support the acquisition cycle: 


a. Products 

• Army Input to Evaluation Plan. For multi-service and joint programs 
where ATEG is not the lead OTA, the AST will provide Army unique 
input to the lead OTA. Their input is documented in the Army Input to 
Evaluation Plan 

• Army Input to Evaluation Report. For multi-service OT where ATEG is not 
the lead, the AST will provide Army unique input to the lead OTA by means 
of a document called Army Input to Evaluation Report. Timelines will be 
documented in ADSS. 

• Acquisition Position Memo - ATEG is required to provide a MR 
Memorandum along with either an OER or an OMAR and Safety 
Gonfirmation to the PM and Fife Gycle Management Genter MR Office in 
support of Type Glassification and MR. The Materiel Release 
Memorandum provided with the OER/OMAR should present an ATEG 
position relative to the proposed materiel release. The memorandum 
should either recommend full materiel release or conditional materiel 
release identifying conditions to be resolved before considering full 
materiel release. 
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• Acquisition Document Review Memo 

• Capabilities & Limitations Report-A report for the Commander informing 
what is known and what is not known about a rapid initiative system. 

• Capabilities & Limitations Report-Update-An update to a 

• Concept In-Process Review ATEC senior level leadership review to obtain 
ATEC leadership approval of the set of tests selected to support the approved 
evaluation strategy (See definition for ESR). 

• Concept In-Process Review - Update 

• Emerging Results Brief - AEC may prepare an Emerging Results Briefing 
(ERB) prior to the CER, if requested by the PM. An ERB is understood to be 
draft in nature and does not negate the need for a CER. The ERB is approved 
by the AST Chair’s Directorate Chief. 

• Early Strategy Review ATEC senior leadership level review to obtain ATEC 
leadership approval of the system evaluation concept developed by the AST. 
The ESR addresses the overall evaluation concept that must be resolved 
before a system can proceed to ERP. The AST will document the approved 
concept in the SEP. The approved test strategy needed to support the 
evaluation strategy is coordinated with the T&E WIPT and presented for 
ATEC leadership approval at the concept in-process review (CIPR) (See 
definition of CIPR). 

• Early Strategy Review - Update 

• OTA Assessment Report - The OAR is not tied to an MDR. It provides an 
evaluation of progress towards meeting system requirements at other times 
than milestones and ERP decision if requested. The OAR may identify 
needed corrective actions; assess readiness for lOT; evaluate the system’s 
logistic supportability and MANPRINT, etc. 

• OTA Evaluation Report - Documents the independent system evaluation 
findings and recommendations regarding a system’s operational ESS and 
safety as well as a system’s mission capability. It is provided at ERP Decision 
Review and is supported by a SAR. The SAR, if required, provides the 
detailed analyses to support the evaluation. 

• OTA Evaluation Report-Update 

• OTA Follow On Evaluation Report Provides additional information on the 
efficacy of corrective actions for system deficiencies found during the lOT. 
OFERs therefore are submitted to decision making officials after the ERP 
decision is made 

• OTA Milestone Assessment Report OMARs provide the decision 
authorities with an independent assessment of the system’s performance and 
operational effectiveness, suitability and survivability at MS B and MS C 
Eow Rate Initial Production (ERIP). Required to be completed by the AST 
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within the E+60 day or no later than Milestone Decision Review (MDR)-45 
day timeframe. 

Safety Confirmation A document issued by AEC that provides the Materiel 
Developer and the decision maker with the test agency’s safety findings and 
conclusions, and that states whether the specified safety requirements have 
been met, includes a risk assessment for hazards not adequately controlled, 
lists any technical or operational safety limitations, and highlights any safety 
problems requiring further testing. The Safety Confirmation may be attached 
to the OER, OAR, or OMAR as applicable. Eor aviation testing, an 
Airworthiness Release does not negate the need for a Safety Confirmation. 

Safety Confirmation Recommendation - Issued to other services, joint 
services, Eoreign Military Sales (EMS), and USSOCOM when requested. 
Recommendations provided by AEC to non-Army organizations will be 
written to the government project sponsor for that item. If the materiel is also 
being fielded to the whole Army, then a SC will be provided by AEC. 

System Evaluation Plan - The SEP documents the ATEC plan for the 
approved integrated system T&E strategy for overall system evaluation. 
The SEP describes the strategy for assessing ESS and evaluating the 
contribution of the system to overall mission capability. The SEP also 
describes the strategy for identifying system capability limitations and 
assessing risks and the potential impact on mission capability. It includes 
refinement of the planned evaluation support to be provided to the 
decision body and the refinement of the test, M&S, and analysis event 
strategies necessary to support the evaluation. 

System Evaluation Plan - Update 

Safety Release = A formal document issued by AEC before any hands-on 
testing, training, use, or maintenance by Soldiers. A Safety Release is issued 
for a specific event at a specified time and location under specific conditions. 
It is a stand-alone document that indicates the system is safe for use and 
maintenance by Soldiers and describes the specific hazards of the system 
based on test results, inspections, and system safety analysis. Operational 
limits and precautions are included. The Safety Release must be available 
prior to start of testing, training, etc. Eor aviation testing, an Airworthiness 
Release does not negate the need for a Safety Release. 

Safety Release Recommendation - Issued to other services, joint services, 
Eoreign Military Sales (EMS), and USSOCOM when requested. 
Recommendations provided by AEC to non-Army organizations will be 
written to the government project sponsor for that item. If the materiel is also 
being fielded to the whole Army, then a SC will be provided by AEC. 

System Analysis Report (SAR). Provides the detailed analysis that supports 
ATEC findings as reported, but in a less restricted time frame than the 
OMAR/OER/OEER. If required, the SAR will be produced by the AEC 60 
days after the OMAR/OER/OEER is completed. The SAR documents the 



analyses that were conducted but not presented within the 
OMAR/OER/OFER. 


• Test & Evaluation Concept Briefing. In order to carry out the Secretary of 
Defense’s responsibilities under Title 10, Section 139, U.S. Code requires 
DOT&E to monitor and advise the Secretary of Defense of the capability 
and resources of the OTA to adequately plan, execute, and report on the 
OT. Within ATEC, the AST fulfills the function of obtaining DOT&E 
approval of the test concept by means of the Test Concept Brief. The briefing 
is provided to the appropriate Deputy Director of DOT&E no later than 180 
days prior to the planned first day of OT (T-180). The intent is to gain early 
DOT&E understanding and approval of the proposed test concept and resolve 
key issues, if necessary, prior to finalizing the OTA TP. 

• Input to Test & Evaluation Master Plan - The TEMP is the basic planning 
document for a system life cycle T&E. The TEMP documents the T&E 
strategy and is developed and initially approved prior to program initiation. 
The TEMP is then updated prior to each subsequent MS and FRP decision 
review thereafter or for a major modification. It is the reference document 
used by the T&E community to generate detailed T&E plans and to ascertain 
schedule and resource requirements associated with a given system. The 
TEMP describes what testing is required, who will perform the testing, what 
resources will be needed, and what the requirements are for evaluation. 

• Test & Evaluation Strategy - The TES integrates all T&E activities 
supporting the program and takes full advantage of existing investments in 
DOD ranges and facilities. The T&E strategy supports the requirements 
and acquisition strategies. It describes how the system concept will be 
evaluated against mission requirements. 

• Requirements documents. JCIDS documents serve as a means for sponsors to 
submit identified capability requirements and capability gaps, along with 
other relevant information, for review and validation.” For materiel solutions, 
the JCIDS documents of interest to T&E are the initial capability document 
(ICD), capability development document (CDD), the capability production 
document (CPD), the urgent operational need (UON), and the joint UON 
(JUON) or joint emergent operational needs (JEON). 
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b. 


Measurement 


Product 

Acronym 

Number 

Produced 

Army Input to Evaluation Plan 

AIAR 

1 

Army Input to Evaluation Report 

AIER 

1 

Acquisition Position Memo 

APM 

31 

Acquisition Document Review Memo 

ADR 

63 

Capabilities & Limitations Report 

C&L Report 

50 

Capabilities & Limitations Report-Update 

C&L Update 

26 

Concept In-Process Review 

CIPR 

15 

Concept In-Process Review - Update 

CIPR- U 

1 

Emerging Results Brief 

ERB 

1 

Early Strategy Review 

ESR 

21 

Early Strategy Review - Update 

ESR- U 

1 

OTA Assessment Report 

OAR 

15 

OTA Evaluation Report 

OER 

17 

OTA Evaluation Report - Update 

OER-U 

1 

OTA Follow On Evaluation Report 

OFER 

0 

OTA Milestone Assessment Report 

OMAR 

20 

Safety Confirmation 

SC 

132 

Safety Confirmation Recommendation 

SCR 

21 

System Evaluation Plan 

SEP 

4 

System Evaluation Plan - Update 

SEP-U 

5 

Safety Release 

SR 

97 

Safety Release Recommendation 

SRR 

10 

Test & Evaluation Concept Briefing 

T&E Concept 

39 

Input to Test & Evaluation Master Plan 

TEMP 

39 

Test & Evaluation Strategy 

TES 

2 


Table 19. AEC FY13 Products with Counts 


AEC established a customer survey program in EY13 with the purpose of 
providing information to formulate short and long term business improvement plans with 
the focus on improving customer satisfaction. The survey focuses on AST proficiency, 
product quality, technical adequacy, timeliness, cost, communication, responsiveness, and 
product satisfaction. A customer survey is sent to the customer for each product (as 

defined in the enclosure) prepared by AEC seven days after product distribution. 
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Currently the survey is sent to the Program Manager only. Quarterly tracking and 
reporting of survey results focusing on trends and systemic issues are presented to the 
AEC leadership. The follow-up actions from the survey results will help improve the 
AEC quality, helpful helpfulness, and timeliness and product satisfaction to the customer 
and to the acquisition process. 

(1). Indicators of performance 

AEC has limited formal metrics established for performance within AEC as it is 
difficult to measure performance in an intellectual environment. AEC relies on real-time 
feedback from decision-makers on the products provided; metrics against the AEC 
Strategic Plan and the common metrics used for resource management. 

ATEC implements the Defense Equal Opportunity Management Institute’s 
Organizational Climate Survey across the command. Military commanders are required 
to conduct a climate assessment within 120 days after assuming command, and at least 
annually thereafter. The climate survey anonymously assesses perceptions of 
organizational effectiveness, equal opportunity, equal employment opportunity, fair 
treatment, and sexual assault prevention and response. The detailed results from the 
survey are not available for analysis; however, the results drive the AEC strategy. 

3. Outcomes 

Outcomes are the intended and unintended consequences of the outputs for the 
stakeholders to include how they are viewed. Outcomes are the “so what” question 
following outputs and can be difficult to measure. Organizations often default to 
measures such as number of reports produced; number of hours worked, timeliness of 
products, etc. 

AEC stakeholders include AEC Employees; Milestone Decision Authority 
(MDA); Program Executive Officers/Program Managers (PEOs/PMs); Director, 
Operational Test & Evaluation (DOT&E), Deputy Assistant Secretary of Defense 
(Developmental Test & Evaluation) (DASD(DT&E); operational units to include the 
individual Soldiers, sailors, air men and marines; and the American taxpayer. 
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The outcomes for AEC workforce include continued employment and job 
satisfaction. Job satisfaction is a fluctuating, measurable consequence of the AEC system. 
High job satisfaction leads to better retention and eases the recruitment needs of the 
organization. 

The cost, schedule and performance of systems managed by the PEOs/PMs can be 
impacted if the products provided by AEC are not timely and accurate. It is well known 
that the costs to fix errors increase as the system matures. Correcting defects later in the 
system development life cycle has been estimated to add from 10 percent to 30 percent to 
the cost of each item (DAU, page 39). As costs and schedules increase, quantities are cut 
and the value to the operational force is reduced. It is imperative that AEC’s insights to 
system performance be provided quickly and accurately. 

T&E results figure prominently in the decisions reached at design and milestone 
reviews. T&E results may not always be favorable. The final decision responsibility lies 
with the decision-maker who weighs the system’s capabilities and shortcomings with the 
acceptable risk. The MDA will be unable to make this judgment without a solid base of 
information provided by T&E. If the information provided by AEC is not timely, systems 
may proceed into operational test prematurely. 

The outcomes for the DOT&E and DASD (DT&E) are similar to those of the 
PEO/PMs. If T&E is not adequate, significant problems in acquisition programs are may 
be discovered during operational testing that should have been discovered in 
developmental testing, potentially causing additional Congressional oversight and 
reporting. 

Injury or death to operational units to include the individual are affected if 
equipment is fielded that is not effective, suitable or survivable. Additionally, equipment 
deemed as non-value will not be used, wasting taxpayers’ dollars. 

Taxpayers are affected if equipment is fielded that is not effective, suitable or 
survivable. An unintended consequence is the waste or misuse of taxpayers’ dollars. 
When/if this occurs and the taxpayers become aware of the situation; the American 
public loses confidence in Army acquisition capabilities. 
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IV. ANALYSIS 


As previously stated, Robert’s OSF model was derived from the basic Inputs, 
Processes and Output model and Nadler and Tushman’s congruence theory of 
organizations (Nadler & Tushman, 1980). It analyzes the components of an organization 
(inputs, throughput, and results) and assesses their congruence in order to draw 
conclusions and make recommendations concerning complex organizational behaviors. 

According to Nadler and Tushman, a critical measurement of effectiveness is the 
alignment or congruence of an organization’s components. Under the systems view of 
organizations, congruence can be defined as the degree to which an organization’s 
components fit together. The basic hypothesis of the congruence theory is that the greater 
the degree of congruence or fit among an organization’s components, the more effective 
the organization will be in achieving its intended strategy (Nadler & Tushman, 1980). 

In the following section an assessment will be made on the congruence between 
the inputs, throughput, and results of the AEG organization. AEG does not have metrics 
in place to analyze these factors objectively. The level of congruence between factors was 
subjectively assessed based in the data and information presented in Ghapter III. The 
terms that will be used to characterize the level of congruence between factors are “NA” 
(no relationship exists and does not to exist), “weak” (needs major improvement), 
“average” (potential of fine-tuning), and “strong” (sustain as-is). 

A. CONGRUENCE BETWEEN INPUT AND THROUGHPUT FACTORS 

First, congruence between inputs and throughput is evaluated to determine “To 
what extent do the inputs affect the throughput?” 

1. Environment-Tasks/Jobs 

The relationship between the environment and tasks/jobs was analyzed to 
determine the congruency of the factors. 
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a. Environment (Political)-Task/Jobs 

The relationship between the Environment (Political) and Tasks/Jobs (Figure 29) 
is assessed as “Average.” 

Environment (Political) - Tasks/Jobs Relationship 


Congressional Reporting 


Task/Jobs 

• System Acq Report (SAR) 


• Evaluate effectiveness, suitability, 

•DOT&E Annual Report 


survivability independently 

• Beyond LRIP Report 


• Verify system safety 

• Foreign Comparative Test 


• Direct test strategy 

• DASD (DT&E) Annual Report 






Figure 29. Environmental (Political)-Tasks-Jobs Relationship 

AEC provides information included in the reports submitted to Congress. The 
task/jobs do support the requirements of the Congressional Reports, however, fine tuning 
of the supporting processes and procedures is required based on the FY13 DOT&E and 
DASD(DT&E) Annual Reports. The fine tuning requirements are addressed later in the 
paper. 


b. Environment (Economic)-Task/Jobs 

The relationship between budget and the tasks assigned to AEC (Figure 30) were 
analyzed. There is no direct relationship between the budget and the tasks assigned to 
AEC and therefore assessed as “NA.” 
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Environment (Economic) - Tasks/Jobs Relationship 



Figure 30. Environmental (Economic)-Tasks-Jobs Relationship 

AEG must perform the tasks regardless of the external economics to include 
budget and increase of employee fringe benefit. 

c. Environment (Social Pressures)-Task/Jobs 

There is no relationship between the social pressures and the tasks assigned to 
AEG (Eigure 31) and therefore assessed as “NA.” 

Environment (Social Pressures)-Tasks/Jobs Relationship 




Eigure 31. Environmental (Social Pressures)-Tasks-Jobs Relationship 
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The social pressures from the external environment do not determine the 
tasks/jobs AEC is required to perform. 

d. Environmental (Technological)-Task/Jobs 

There is no relationship between the technological factor and the tasks assigned to 
AEC (Eigure 32) and is therefore assessed as “NA.” 


Environment (Technological)-Tasks/Jobs Relationship 


Cybersecurity 



Eigure 32. Environmental (Technological)-Tasks-Jobs Relationship 

Although AEC may need to adjust the workflow required to execute the task/jobs, 
the technological aspect of the external environmental does not impact the task/jobs that 
AEC is directed to execute. 

2. Environment-Technology 

a. Political-Technology 

The relationship between the environment (political) and technology (workflow) 
(Eigure 33) was analyzed and there is a strong congruency identified. 
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Environment (Political)- Technology (Workflow) 



Figure 33. Environmental (Political)-Technology (Workflow) 


It is assumed that AEG provides information to support the SAR that is of value 
as there is no evidence to support otherwise. The EY13 DOT&E Annual report discusses 
problem discovery affecting OT&E. The report states “ Adequate developmental and 
operational testing are essential for determining whether systems provide an effective, 
suitable, and survivable warfighting capability to our Soldiers, Sailors, Airmen, and 
Marines.” The report further states that Congress has expressed concern that problems are 
not found early enough to prevent cost and schedule increases or is a “fatal flaw” that 
cannot be fixed. DOT&E offers several solutions to mitigate this issue to include: allow 
adequate time for developmental testing and resolve issues found during DT; 
operationally focused DT; TRADOC provide earlier concept of operations and 
employment for better test design; improved requirements documents and contract 
language to include not only performance specifications, but mission capabilities. 
Additionally, DOT&E addressed cybersecurity vulnerabilities discovered during OT. The 
DASD(DT&E) Annual Report provided guidance on initiatives such as “Shift Eeft” 
which supports DOT&E’s recommendations on robust DT. The findings in these reports 
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as well as the upcoming changes to DOD 5000.02 will cause AEC to adjust how the 
assigned tasks are performed. 

b. Environmental (Economic)-Technology 

The relationship between the economic factor and technology (workflow) (Figure 
34) was analyzed and there is not a direct correlation and therefore is assessed as “NA.” 
There are second and third order effects regarding the resourcing/manning of the 
workflow that are addressed later in the paper. 


Environment (Economic)-Technology (Workflow) 






Figure 34. Environmental (Ecnomic)-Technology (Workflow) 


c. Environment (Social)-Technology - 

The relationship between the economic factor and technology (workflow) (Figure 
35) was analyzed and there is no relationship. 
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Environment (Social Pressures)-Technology (Workflow) 



Figure 35. Environmental (Soeial Pressures)-Teehnology (Workflow) 
d. Technological-Technology 

The relationship between the technological factor and technology (workflow) was 
analyzed (Figure 36) and there is a weak congruency. 


Environment (Technological) - Technology(Workflow) 



Figure 36. Environmental (Technological)-Technology (Workflow) 
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The increased emphasis on cybersecurity, especially in the system of system 
context, and support to the “Shift Left” initiative will cause more effort in earlier in the 
program. This increased effort will result in more information at Milestone A. The NIL 
and experimentation efforts cause deviations in ABC’s T&E processes. 

3. Environment - Structure 

a. Political-Structure 

The relationship between the political factor and structure was analyzed (Figure 
37) and there is a strong congruency. 


Environment (Political)- Structure 



Congressional Reporting 

• System Acq Report (SAR) 
•DOT&E Annual Report 

• Beyond LRIP Report 

• Foreign Comparative Test 

• DASD (DT&E) Annual Report 
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Figure 37. Environmental (Political)-Structure 

AEC is structured around the Army’s warfighting functions. This structure is 
attractive to DOT&E as it supports the operational effectiveness, suitability and 
survivability. DASD(T&E) is championing an effort to “operationalize” DT. The AEC 
structure supports that initiative as well. 
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b. 


Economic-Structure 


The relationship between the political factor and structure (Figure 38) was 
analyzed and there is an average congruency. 


Environment (Economic)- Structure 
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Figure 38. Environmental (Economic)-Structure 

With the budgets decreasing, the structure may become too “management heavy.” 
There may be different organizational constructs that may reduce the number of 
directorates and divisions to mitigate the risk resulting from the budget reductions. 

c. Social Pressures- Structure 

The relationship between the social pressure factor and structure (Eigure 39) was 
analyzed and there is none. This pair is assessed as “NA.” 
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Environment (Social Pressures)-Structure 



Figure 39. Environmental (Social Pressures)-Structure 
d. Technological-Structure 

The relationship between the technological factor and structure was analyzed 
(Figure 40) and there is no correlation and is assessed as “NA.” 
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Figure 40. Environmental (Technological)-Structure 
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4. Environment - People 

a. Political-People 

The relationship between the political factor and people was analyzed (Figure 41) 
and there are second and third order effects regarding budget and acceptance of products; 
however there is not a direct relationship and therefore assessed as “NA.” 


Environment (Political)- People 


Congressional Reporting 

• System Acq Report (SAR) 
•DOT&E Annual Report 

• Beyond LRIP Report 

• Foreign Comparative Test 

• DASD (DT&E) Annual Report 



Figure 41. Environmental (Political)-People 
b. Economic-People 

The relationship between the economic factor and people was analyzed (Figure 
42) and there is congruency and assessed as “Average.” 
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Environment (Economic)- People 



Figure 42. Environmental (Economic)-People 

Most Department of Defense civilians, to include AEG, were furloughed for 11 
days between July and September 2013, placing employees in a temporary non-duty, 
non-pay status because of lack of funds. This represents a 20 percent reduction in pay 
during that period. Major budgetary shortfalls drove the basic furlough decision. 

DOD Civilian Personnel Eringe Benefits cost continue to increase. The benefit 
cost is combined with the civilian labor costs and paid for by AEC. As the benefit cost 
increases, there is more of a demand on the budget. Civilian salaries will have limited 
increases. 

As the budget and authorizations continue to drawdown, AEC will need to closely 
continue to manage hiring actions to balance authorization and workload. 

c. Social Pressures-People 

The relationship between the social pressures and people was analyzed (Eigure 
43) and there is no correlation and therefore assessed as “NA.” 
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Environment (Social Pressures)- People 




Figure 43. Environment (Social Pressures)-People Relationship 


Social pressures may impact the DOD and Army levels, which will then impact 
AEG. However, AEG ‘s people (manpower authorizations and the on-board personnel) 
are not impacted by social pressures. 

d. Technological-People 

The relationship between the technological factor and people was analyzed 
(Eigure 44) and there is an average congruency and therefore assessed as “Average.” 
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Figure 44. Environment (Technological)-People Relationship 

AEC will need to closely continue to manage hiring actions to balance 
authorization with the new skill mix required for future missions resulting from 
technologies emerging from Army Science and Technology. Additionally, as the 
cybersecurity mission matures, AEC may need additional expertise and an increase of 
authorizations for military and civilian. 

5. Environment-Process/Subsystems 

The relationship between the environmental factors and process/subsystems was 
analyzed. 


a. Political-Process/S ub systems 

The relationship between the environmental (political) factor and 
process/subsystems (Eigure 45) was analyzed and assessed as a strong congruency due 
the heavily regulated processes supporting financial and human resource management. 
ATEC provided additional guidance in acquisition and contracting by emplacing policy 
to mitigate conflict of interest. 
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Environment (Political)- Process/Subsystems 



Congressional Reporting 

• System Acq Report (SAR) 
•DOT&E Annual Report 

• Beyond LRIP Report 

• Foreign Comparative Test 

• DASD (DT&E) Annual Report 




Figure 45. Environment (Political)-Process/Subsystems Relationship 

The cost of the AEG civilian workforce labor is direct funded (not 
Customer/Program Manager reimbursed). AEG currently uses contractors to augment the 
current civilian and military workforce; the products provided are not the conclusions of 
the final evaluation product but may include data reduction and analysis, data bases (and 
management) and data produced by models and simulations. The efforts performed by 
contractors are 100% reimbursed by the customer. AEG civilian and military personnel 
provide technical and administrative oversight and control of all contractor efforts. This 
oversight and control function is inherently governmental and is not appropriate for 
contractors to perform. 

Title 10 §2399 states the Director of Operational Test and Evaluation of the 
Department of Defense approves (in writing) the adequacy of the plans (including the 
projected level of funding) for operational test and evaluation to be conducted in 
connection with that program. Title 10 further states the Director shall prepare a report 
stating his opinion whether the test and evaluation performed were adequate; whether the 
results of such test and evaluation confirm that the items or components actually tested 
are effective and suitable for combat; and additional information on the operational 
capabilities of the items or components that the Director considers appropriate based on 
the testing conducted. 
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There is no relationship between environmental (political) and communications, 
information, planning and decision making. 

b. Economic-Process/Subsystems 

The relationship between the environmental (economic) factor and 
process/subsystems (Figure 46) was analyzed and there is average congruency. 


Environment (Economic)- Process/Subsystems 



Figure 46. Environment (Economic)-Process/Subsystems Relationship 

As budgets decrease, the demand on the resources increases. This results in 
additional reporting and more detailed tracking of budgets. Human resource management 
processes are stressed as AEG manages workforce reshaping by offering VERA-VSIP 
and other tools. Communications become more important to ensure the workforce is 
informed of resource decisions so they can plan for their futures. Planning and decision¬ 
making processes value increases as the decisions made will have lasting effect on the 
mission and the workforce. 

c. Social-Process/Subsystems 

The relationship between the environmental (social pressures) factor and 
process/subsystems (Eigure 47) was analyzed and there is weak congruency due to the 
communications process/subsystems. 
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Environment (Social Pressures)- Process/Subsystems 



Figure 47. Environment (Social Pressures)-Process/Subsystems 

Relationship 

The heavily regulated processes of Resource Management, Human Resources and 
Acquisition & Contracting are not influenced by social pressures at the AEC level. These 
processes may be influenced at the DOD and Army levels, which will eventually modify 
the processes within AEC and ATEC. Social pressures may put an additional demand on 
the communications processes as shown with the Palantir situation to ensure the AEC 
workforce is informed on the processes and procedures for speaking with the media. 

d. Technological-Process/Subsystems 

The relationship between the environmental (technological) factor and 
process/subsystems (Eigure 48) was analyzed and there is no correlation and therefore 
assessed as “NA.” 
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Environment (Technological)- Process/Subsystems 



Figure 48. Environment (Technological)-Process/Subsystems 

Relationship 


6. Key Success Factors-Task/Jobs 

The relationship between the Key Success Factors and Task/Jobs (Figure 49) was 
analyzed and there is a strong congruency. 


Key Success Factors - Tasks/Jobs 




Task/Jobs 

• Goal Attainment 


• Evaluate effectiveness, suitability, 
survivability independently 

• Resource Utilization 


• Verify system safety 

• Direct test strategy 

• Adaptability 




Figure 49. Key Success Factors-Task/Jobs 

The AEG Strategic Plan Goal 2: “Continue to Improve Product Value to our 
Customers” focuses on improving the value of AEG products to the decision-makers. 
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Initiatives are built to support this goal with metrics to assess progress towards attaining 
the goal. 

7. Key Success Factors-Technology 

The relationship between the Key Success Factors and Task/Jobs (Figure 50) was 
analyzed and there is a strong congruency. 


Key Success Factors - Technology (Workflow) 



Figure 50. Key Success Factors-Technology (Workforce) Relationship 

The AEC Strategic Plan Goal 2: “Continue to Improve Product Value to our 
Customers” focuses on improving the value of AEC products to the decision-makers. 
Initiatives are built to support this goal with metrics to assess progress towards attaining 
the goal. Additionally, AEC Strategic Plan Goal 3: Ensure the organization is structured 
for efficient operations supports the resource utilization and adaptability factors. 

8. Key Success Factors-Structure 

The relationship between the Key Success Factors and Structure (Figure 51) was 
analyzed and there is an average congruency assessment. The actual structure of AEC 
does impact goal attainment and resource utilization. The current structure is composed 
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of 10 directorates with 46 divisions. With the reductions to budget, the number of 
directorates and divisions may decrease resulting in less management and more 
manpower performing the work. This supports all three key success factors. Goal 3 of the 
AEG Strategic Plan focuses on efficient operations and includes an objective “Creating a 
flexible organization to respond to changing workload environment” 


Key Success Factors - Structure 



Figure 51. Key Success Factors-Structure Relationship 

9. Key Success Factors-People 

The relationship between the Key Success Factors and People (Figure 52) was 
analyzed and there is strong congruency. 


Key Success Factors - People 



Figure 52. Key Success Factors-People Relationship 
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The People factor is strongly related to Goal Attainment, Resource Utilization and 
Adaptability. The main thrust of Goal 1 is centered on the AEG workforce. The goal 
states “Organization that is a Great Place to Work” with objectives of “Maintain an 
Organization of Talented Professionals”; “Raise Workforce Credentials and 
Certifications”; “Ensure good communications and transparency”; “Increase team work”; 
and “Improve workforce quality of life affect the AEG workforce.” 

10. Key Success Factors-Process/Subsystems 

The relationship between the Key Success Eactors and Process/Subsystems 
(Eigure 53) was analyzed and the congruency is assessed as “Average.” 


Key Success Factors-Process/Subsystems 



Eigure 53. Key Success Eactors-Process/Subsystems Relationship 

AEC’s Strategic Plan third objective focuses on the organization’s operations. 
The processes of Resource Management and Human Resources are impacted. Currently 
processes with the ATEC staff will need to be “smoothed.” Ad-hoc and unstable 
processes cause inefficiency as the interfaces between AEG and ATEC HQ are not well 
defined. Additionally, no documented process exists to reassign resources (military and 
civilian manpower) to balance workload. 
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11. System Direction-Task/Jobs 

The relationship between the System Direction and Process/Subsystems (Figure 
54) was analyzed and there is strong congruency. 


System Direction-Tasks/Jobs Relationship 


Mandate? 


Task/Jobs 

Values? 


• Evaluate effectiveness, suitability, 

Mission? 


survivability independently 

Strategic Issues? 


• Verify system safety 

Vision? 


• Direct test strategy 

Goals? 



Strategies? 




Figure 54. System Direction-Tasks/Jobs Relationship 

As described in earlier sections, the AEC mission is derived from the mandates 
and the tasks/jobs are derived from the mission. There is a strong linkage between the 
system direction and the tasks/jobs. The Army Values provide the foundation for the 
leadership required to perform the tasks/jobs. AEC Strategic Plan goals and objectives 
support continuous improvement (effectiveness and efficiency) of the tasks/jobs defined. 
The AEC vision statement incorporates the system direction and the tasks/jobs. 

12. System Direction-Technology 

The relationship between the System Direction and Technology (Eigure 55) was 
analyzed and there is strong congruency. 
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System Direction - Technology (Workflow) 



Figure 55. System Direction Factors-Technology (Workforce) Relationship 

There is a strong linkage between the system direction and the workflow. The 
Army Values provide the foundation for the leadership required to execute the workflow. 
AEC Strategic Plan goals and objectives support continuous improvement (effectiveness 
and efficiency) of the workflow defined. The AEC vision statement incorporates the 
system direction and the best practices required to have an efficient and effective 
workflow. 


13. System Direction-Structure 

The relationship between the System Direction and the Structure (Eigure 56) was 
analyzed and there is average congruency. 
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System Direction - Structure 


Mandate? 
Values? 
Mission? 
Strategic Issues? 
Vision? 
Goals? 
Strategies? 



Figure 56. System Direction-Structure Relationship 

The mandates direct that AEC provide evaluations focusing on the operational 
effectiveness, suitability and survivability. ABC’s structure is almost perfectly aligned to 
support the mandates. The directorates of C2ED, BED, lED, MAED, MGED and SED 
focus on the effectiveness of the weapon systems assigned. SVED focuses on the 
survivability evaluations. BSD and IBS perform the suitability evaluations. AEC may 
consider consolidating BSD (RAM) and IBS to support the integrated suitability 
evaluation required by the mandates. 

14. System Direction-People 

The relationship between the System Direction and People (Eigure 57) was 
analyzed and there is average congruency. 
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System Direction- People 


Mandate? 
Values? 
Mission? 
Strategic Issues? 
Vision? 
Goals? 
Strategies? 



■ AuthoruJiiom 


Figure 57. System Direction-People Relationship 

The mandates direct the initiatives such as increased cybersecurity during DT, use 
DoE on all ACAT I programs and integrate suitability (operationalize RAM using the 
ILS as the mechanism). This drives AEG to analyze the current skills on-board and 
determine the gaps in the on-board strengths. This gap analysis will drive recruitment 
strategies and/or training plans. 

The ATEC-mandated Control Points currently support affordability of the AEC 
workforce, however, in some positions, the workforce would benefit from the General 
Service (GS) system. Eor example, an NH-04 is a GS-14 equivalent. The salary 
opportunity for the NH-04 would be $129,519 as a GS employee instead of capped at 
$110,734. 

15. System Direction-Process/Subsystems 

The relationship between the System Direction and Process/Subsystems (Eigure 
58) was analyzed and there is strong congruency. 
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System Direction- Process/Subsystems 


Mandate? 
Values? 
Mission? 
Strategic Issues? 
Vision? 
Goals? 
Strategies? 







Figure 58. System Direction-Process/Subsystems Relationship 

The system direction impacts the process/subsystems of Financial Management, 
Measurements and Controls by directing a strong management internal control program. 
AEC receives most of the staff services from ATEC HQ. ATEC HQ has a robust MICP, 
for their staff processes; however AEC does not have a formal program for their internal 
processes. AEC’s internal processes are currently inconsistent and ad-hoc. Documented, 
standardized, consistent, efficient, effective, enforced, and streamlined processes are 
required to ensure effective and efficient operations. 

The Human Resource Management process/subsystem is impacted by the 
mandates, goals and strategies. The HR dimension is heavily regulated in the federal 
government, and therefore there is very little flexibility in the recruitment, selection, 
rotation, promotion or termination of the workforce. Retirement is an individual’s option, 
there is no mandatory retirement age in federal service. 

The ATEC Policy Bulletin 2-11, Organizational Conflicts of Interest (OCI) 
Involving Contractors in Support of ATEC Test and Evaluation (T&E) implements 
policy and procedures to ensure the avoiding violations of Title 10, United States Code, 
Section 2399(d). 
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Organizations should communicate their values and the behavioral expeetations 
that go along with those values to their employees. AEC’s eommunications proeesses are 
not as robust as they eould be in today’s world of soeial media. AEC’s geographieally 
dispersed lay-out eauses gaps in the eurrent eommunieation processes. 

16. Summary of Congruence between Input and Throughput Factors 

Table 20 shows the summary of eongruenee between input and throughput faetors 
that determine the extent of input faetors affeeting throughput faetors. 


Design Factors 

Task/Jobs 

Technology 

Structure 

People 

Process/ 

Subsystem 

Envi ronment 

(Political) 

Average 

Strong 

Strong 

NA 

Strong 

Envi ronment 

(Economic) 

NA 

NA 

Average 

Average 

Average 

Envi ronment 

(Social) 

NA 

NA 

NA 

NA 

Weak 

Envi ronment 

(Technological) 

NA 

Weak 

NA 

Average 

NA 

Key Success Factors 

Strong 

Strong 

Average 

Strong 

Average 

System Direction 

Strong 

Strong 

Average 

Average 

Strong 


Table 20. Summary of Congruenee of Input-Throughput Eaetors 


The areas for improvement for the throughputs in relation to the inputs are those 
assessed as “Weak” and “Average.” Areas assessed as “weak” are targets of 
improvement; those areas assessed as “Average” may require fine-tuning, bindings and 
reeommendations for improvement are addressed in Chapter V. 

B. CONGRUENCE OF THROUGHPUT FACTORS 

A key underlying faetor of the OSE model is that organizations possess a greater 
ability to eontrol faetors within the throughput as opposed to inputs or results. 
Congruenee among the throughput faetors is evaluated to determine “To what extent do 
the throughput factors affect each other?” 
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1 . 


Tasks/Jobs-Technology 


The relationship between the Tasks/Jobs and Technology (Figure 59) was 
analyzed and there is strong congruency. 

Tasks/Jobs - Technology Relationship 


Task/Jobs 

• Evaluate effectiveness, suitability, 
survivability independently 

• Verify system safety 

• Direct test strategy 



Figure 59. Task/Jobs-Technology Relationship 

Purposely, the workflow does align strongly to the tasks/jobs. There may be fine- 
tuning of the workflow as the changes to the DOD 5000.02 are published. 

2. Tasks/Jobs-Structure 

The relationship between the Tasks/Jobs and Structure (Figure 60) was analyzed 
and there is strong congruency. The organization is structured to support effectiveness, 
suitability and survivability. System safety and directing test strategy are executed by 
subject matter experts assigned to each effectiveness directorate. 
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Figure 60. Tasks/Jobs-Structure Relationship 

3. Task/Jobs-People 

The relationship between the Tasks/Jobs and People (Figure 61) was analyzed and 
there is a strong congruency. The current manning for each directorate appears to be 
adequate to execute AFC’s assigned tasks. 


Tasks/Jobs - People Relationship 



■ Authormttons 


Figure 61. Tasks/Jobs-People Relationship 
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The knowledge, skills and abilities of the AEC workforce are aligned to the 
mission. There is fine-tuning of certain skill sets as the weapon systems become more 
complex and software driven. However, AEC skill set to perform the tasks/jobs is 
aligned. 

4. Task/Jobs-Process/Subsystems 

The relationship between the Tasks/Jobs and Process/Subsystem (Eigure 62) was 
analyzed and there is strong congruency. ATEC HQ as the servicing staff supports AEC 
with adequate resource management, human resource management and contracting 
management support. One area that may need improvement is communications in terms 
of public affairs. 


Tasks/Jobs - Process/Subsystems Relationship 



Eigure 62. Tasks/Jobs-Process/Subsystems Relationship 

5. Technology-Structure 

The relationship between the Technology and Structure (Eigure 63) was analyzed 
and there is an average congruency. The current structure was purposely designed to 
execute the T&E processes, however the ATEC HQ servicing staff concept was not in 
place at the time of design. 
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Technology - Structure Relationship 
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Figure 63. Technology-Structure Relationship 

An example of the incongruent factors between the T&E process and the current 
structure is the tasking process. ATEC HQ Directorates task AEG directorates without 
going through an operations cell. Up to eleven (11) separate mission analyses are 
completed to determine if task should be executed by AEG. This workflow results in 

• inconsistent processes for obtaining an AEG HQ position and sometimes 
omitting AEG HQ from the decision-making process 

• occurrences of duplicative tasks from the ATEG HQ staff 

• no reuse of information from prior tasks 

6. Technology-People 

The relationship between the Technology and People (Eigure 64) was analyzed 
and there is strong congruency. 
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Technology - People Relationship 




Figure 64. Technology-People Relationship 

The T&E processes are supported by the current workforce. 

7. Technology-Process/Subsystems 

The relationship between the Technology and Process/Subsystems (Figure 65) 
was analyzed and there is strong congruency. ATEC HQ provides staff services to AEG 
that support the workflow. 


Technology - Process/Subsystems Relationship 



Eigure 65. Technology-Process/Subsystems Relationship 
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8. Structure-People 

The relationship between the Structure and People (Figure 66) was analyzed and 
there is strong congruency. 


Structure - People Relationship 
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Figure 66. Structure-People Relationships 

9. Structure-Process/Subsystems 

The relationship between the Structure and Process/Subsystems (Figure 67) was 
analyzed and there is strong congruency. ATEC’s servicing staff processes remain stable 
regardless of AEG structure, due to the highly regulated nature of the processes. 
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structure - Process/Subsystems Relationship 




Figure 67. Structure-Process/Subsystems Relationship 

10. People-Process/Subsystems 

The relationship between the People and Process/Subsystems (Figure 68) was 
analyzed and there is strong congruency. 


People - Process/Subsystems Relationship 



Figure 68. People-Process/Subsystems Relationship 
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11. Summary of Congruence of Throughput Factors 

Table 21 shows the summary of congruence between the throughput factors that 
determine the extent of the throughput factors affecting each other. 


Design 

Factors 

Task/Jobs 

Technology 

Structure 

People 

Process/ 

Subsystems 

Task/Jobs 


Strong 

Strong 

Strong 

Strong 

Technology 

Strong 


Average 

Average 

Strong 

Structure 

Strong 

Average 


Strong 

Average 

People 

Strong 

Average 

Strong 


Strong 

Process/ 

Subsystems 

Strong 

Strong 

Average 

Strong 



Table 21. Summary of Congruence of Throughput Factors 

The primary areas for fine-tuning of the throughput factors where congruency is 
assessed as “Average.” Findings and recommendations are addressed in Chapter V. 

C. CONGRUENCE OF THROUGHPUT FACTORS AND RESUUTS 

While the inputs outline strategic direction and the design factors describe the 
structure and the implementation of strategy, results are a way to evaluate the success of 
the fit of these variables in regards to achieving the goals. The results can best be 
characterized by defining the outputs, or measurement of the organization’s performance 
as well as the outcomes, or consequences of the congruence of the variables, both 
intended and unintended. Inputs are transformed to results through the Throughput or 
design factors. To answer the question “To what extent do the throughput factors affect 
the results?” each design factor was compared to the result to determine level of 
congruence. 

1. Tasks/Jobs-Culture 

The relationship between the Task/Jobs and Culture (Figure 69) was analyzed and 
there is no relationship and therefore is assessed as “NA.” 
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Tasks/Jobs-Culture Relationship 



Figure 69. Tasks/Jobs-Culture Relationship 

2. Tasks/Jobs-Output 

The relationship between the Task/Jobs and Output (Figure 70) was analyzed and 
there is strong congruency. However, AEG produces numerous products with the same 
information; only the document name and the document format differ. 



Figure 70. Tasks/Jobs-Output Relationship 
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3. 


Tasks/Jobs-Outcomes 


The relationship between the Task/Jobs and Outcomes (Figure 71) was analyzed 
and there is strong congruency. 


Tasks/Jobs-Outcomes Relationship 



Figure 71. Tasks/Jobs-Outcomes Relationship 

4. Technology-Culture 

The relationship between the Technology and Culture (Figure 72) was analyzed 
and there is no relationship and therefore assessed as “NA.” 


Technology-Culture Relationship 



ISJUIMY 



Figure 72. Technology-Culture Relationship 
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5. 


Technology-Output 


The relationship between the Technology and Output (Figure 73) was analyzed 
and there is strong congruency. The T&E processes exist solely to product the output. 


Technology-Output Relationship 



Figure 73. Technology-Output Relationship 


6. Technology-Outcomes 

The relationship between the Technology and Output Outcomes (Figure 74) was 
analyzed and there is strong congruency. 


Technology - Outcomes Relationship 



Figure 74. Technology-Outcome Relationships 
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7. 


Structure- Culture 


The relationship between the Structure and Culture (Figure 75) was analyzed and 
there is no relationship and therefore assessed as “NA.” 


Structure-Culture Relationship 
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Figure 75. Structure-Culture Relationship 

8. Structure-Output 

The relationship between the Structure and Output (Figure 76) was analyzed and 
there is strong congruency. 
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Structure - Output Relationship 
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Figure 76. Structure-Output Relationship 

The structure supports the product development, which focuses on the 
effectiveness, suitability and survivability evaluation of assigned weapon systems. 
However, combining ILS and BSD may be considered to integrate suitability within a 
directorate. 


9. Structure-Outcomes 

The relationship between the Structure and Outcomes (Figure 77) was analyzed 
and there is strong congruency. 
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Structure - Outcomes Relationship 
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Figure 77. Structure-Outcomes Relationship 

10. People-Culture 

The relationship between the People and Culture (Figure 78) was analyzed and 
there is average congruency as the civilian and military cultures occasionally clash due to 
lack of understanding of the two cultures. 


People-Culture Relationship 




Figure 78. People-Culture Relationship 
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11. People-Output 

The relationship between the People and Output (Figure 79) was analyzed and 
there is strong congruency. ABC’s hybrid workforce of scientists and engineers, 
Acquisition Corps officers and the operational force produce information required by the 
decision makers. 


People-Output Relationship 
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Figure 79. People-Output Relationship 

12. People-Outcomes 

The relationship between the People and Outcomes (Figure 80) was analyzed and 
there is strong congruency. The ABC workforce represents the stake holders; from 
technical engineers assessing performance, to the Acquisition Corps officers managing 
cost, schedule and performance to the operational force providing the voice of the Soldier 
in the evaluations. 
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People - Outcomes Relationship 



Figure 80. People-Outcomes Relationship 

13. Processes-Culture 

The relationship between the Processes/Subsystems and Culture (Figure 81) was 
analyzed and there is average congruency. 


Processes/Subsystems - Culture Relationship 



Figure 81. Processes/Subsystems-Culture Relationship 


131 












The bureaucracy of the processes conflicts with the military culture at times (ie. 
results driven vice process driven). 

14. Processes-Output 

The relationship between the Processes/Subsystems and Output (Figure 82) was 
analyzed and there is average congruency. 


Processes/Subsystems - Output Relationship 



Figure 82. Processes/Subsystems-Output Relationship 

Some of the internal staffing processes are impacting the output. Evaluators lose 
time to evaluate when staffing takes a long time. 

15. Processes-Outcomes 

The relationship between the Processes/Subsystems and Outcomes (Figure 83) 
was analyzed and there is average congruency. 
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Processes/Subsystems - Outcomes Relationship 



Figure 83. Processes/Subsystems-Outcomes Relationship 

The biggest impaet of the proeess/subsystems is the AEC workforee. Many of the 
internal proeesses are ad-hoe and unstable. These proeesses need to be fine-tuned, 
doeumented and implemented. 

16. Summary of Congruence of Throughput Factors to Results 

In Table 22, the summary of eongruenee between the throughput faetors and 
results is shown. 


Design 

Culture 

Outputs 

Outcomes 

Factors 

Task/Jobs 

NA 

Strong 

Strong 

Technology 

NA 

Strong 

Strong 

Structure 

NA 

Strong 

Strong 

People 

Average 

Strong 

Strong 

Process/ 

Average 

Average 

Average 

Subsystems 





Table 22. Summary of Congruenee of Throughput Faetors and Results 
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The primary areas for fine-tuning of the throughput factors where congruency is 
assessed as “Average.” Findings and recommendations are addressed in Chapter V. 

D. SUMMARY OF ANALYSIS 

Table 23 is a summary of congruency for all factors in relation to the throughput 

factors. 


Design Factors 

Task/Jobs 

Technoiogy 

Structure 

Peopie 

Process/ 

Subsystem 

Environment 

(Poiiticai) 

Average 

Strong 

Strong 

NA 

Strong 

Environment 

(Economic) 

NA 

NA 

Average 

Average 

Average 

Environment 

(Sociai) 

NA 

NA 

NA 

NA 

Weak 

Environment 

(Technoiogicai) 

NA 

Weak 

NA 

Average 

NA 

Key Success Factors 

Strong 

Strong 

Average 

Strong 

Average 

System Direction 

Strong 

Strong 

Average 

Average 

Strong 

Task/Jobs 

NA 

Strong 

Strong 

Strong 

Strong 

Technoiogy 

Strong 

NA 

Average 

Average 

Strong 

Structure 

Strong 

Average 

NA 

Strong 

Average 

Peopie 

Strong 

Average 

Strong 

NA 

Strong 

Process/Subsystem 

Strong 

Strong 

Average 

Strong 

NA 

Cu i tu re 

NA 

NA 

NA 

Average 

Average 

Outputs 

Strong 

Strong 

Strong 

Strong 

Average 

Outcomes 

Strong 

Strong 

Strong 

Strong 

Average 


Table 23. Summary of Congruence of All Factors with Throughputs 


Focus should be placed on the throughput factors where the congruency is 


assessed as “weak” or “average” with the number of “counts” of “weak” or “average” 


driving the priority. Table 24 shows the summary of “counts” by throughput design 


factors. 
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Design Factors 

Task/Jobs 

Technology 

Structure 

People 

Process/ 

Subsystem 

Counts 

NA 

5 

4 

4 

3 

2 

Strong 

8 

7 

5 

6 

5 

Weak 

0 

1 

0 

0 

1 

Average 

1 

2 

5 

5 

6 

Total of Weak & Average 

1 

3 

5 

5 

7 


Table 24. Summary of “Counts” by Throughput Design Factors 


In order of impact, ABC’s effectiveness and efficiency may be improved by 
changes to process/subsystems, structure, people, structure and task/jobs. 
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V. FINDINGS AND RECOMMENDATIONS FOR AEC 


A. INTRODUCTION 

The purpose of this joint applied project was to determine if an organization 
system analysis could be used to provide baseline and key information to leaders. The 
project applied the OSF model to identify improvements in ABC’s efficiency and 
effectiveness. This chapter presents findings and recommendations for applying the OSF 
model to AEC. 

B. FINDINGS AND RECOMMENDATIONS 

1. Finding I. 

Organization system analysis using the OSF model was successful in providing a 
baseline and key information required to design AEC for the future. 

Recommendations: 

a. Continue using the OSF to identify future improvements. 

b. Focus on the factors that are within AFC’s control to change (ie. 
throughput factors). 

c. Focus on the factors with the greatest improvement potential. 

2. Finding 2. 

AEC achieves a “fairly strong” level of congruence between the inputs, 
throughputs and results. However, there are two areas where congruency amongst the 
factors is assessed as “weak” and 19 areas where congruency amongst the factors is 
assessed as “average” (see Tables 21 and 22). 

Recommendations: 

a. Establish organizational information program to ensure the AEC 

workforce is reminded of policies and procedures regarding interfacing 
with the media. (Environment(Social)-Process/Subsystem) 
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b. Establish processes and procedures for cybersecurity evaluation and 
other developmental test initiatives. (Environmental (Technological)- 
Technology) 

c. Emphasize the importance of experimental design and other statistical 
methods to ensure test adequacy. (Environment (Political)-Task/Jobs) 

d. Emphasize the importance of developmental test & evaluation to ensure 
systems are ready for operational test. (Environment (Political)- 
Task/Jobs) 

e. Establish an AEC coordination cell for managing tasks and AEC 
corporate efforts. 

f. Establish an effective tasking and task management system. (Structure- 
Technology) 

g. Revisit AEC Control points. (System Direction-People) 

h. Develop Knowledge, Skills and Aptitudes tailored for the AEC mission. 
(System Direction-People) 

i. Revalidate the number of directorates and divisions. (Economic- 
Structure, Key Success Factors-Structure) 

j. Revalidate the size of the divisions. (Economic-Structure) 

k. Develop process to internally reassign civilian workforce to support 
mission decrease-increase. (Economic-People) 

l. Define skill mix required to support emerging requirements such as 
cybersecurity. (Technological-People) 

m. Develop AEC Communications Plan to ensure workforce is aware of 
impacts of upcoming resource reductions (Economic- 
Process/S ubsy stems). 

n. Develop a formal ‘‘Borrowed Manpower” process to include civilians 
and military to improve AEC operations. (Key Success Factors- 
Process/S ubsy stems) 

o. Consider consolidating Engineering Science Directorate (ESD) and 
Integrated Logistics Support Directorate to support integrated suitability 
evaluations. 

p. Develop the AEC Smartbook documenting the staff processes and 
procedures to “smooth” the interface with the ATEC HQ servicing staff. 
(Processes- Culture, Processes-Outcomes) 
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q. Raise the height of the cubicle farms to allow privacy and quiet. 
(Technology-People). 

r. Develop AEC enterprise reporting (manning, TDA, etc). 

s. Establish a Knowledge Management Program; balancing the “need to 
share” and “need to know” 

t. Develop and train conflict resolution processes and procedures. 

u. Establish workshops for military in the “science & technology office 
environment”; similar to the “greening” of civilians. 

Recommendations regarding errors in the authoritative systems include: 

a. Review TDA for accuracy in Acquisition Corps designation. 

b. Update all civilian position descriptions to reflect the current 
requirements to enable workforce shaping (recruitment as well as 
VERA-VSIP). 

Although this research was successful in analyzing AEC as a system, many of the 
recommendations warrant dedicated and more in-depth quantitative analysis or 
consideration from different perspectives. 
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VI. CONCLUSIONS 


In the wake of sequestration, the Army is faced with the daunting task of ensuring 
organizations are structured to properly respond to growing demands. In today’s 
operating environment where resources are diminishing, workload is stable and business 
practices are scrutinized, it is important for organizations to proactively adapt to the 
changes in the external environments. 

The Organizational Systems Framework model used for this Joint Applied Project 
served as an excellent diagnostic tool to identify areas for improvements resulting in 
increased efficiency and effectiveness. Applying the model led to a comprehensive report 
on current activities with recommendation for changes in the future. Although this 
research was successful in analyzing AEG as a system, many of the findings, 
recommendations, and conclusions drawn in this paper warrant dedicated and more in- 
depth quantitative analysis or consideration from different perspectives. 
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